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Overview of TEAC accreditation 



In the TEAC accreditation process, the program’s self-study document is either an Inquiry 
Brief {ior those pursuing accreditation status) or an Inquiry Brief Proposal {tor those pursu- 
ing initial accreditation status). TEAC accredits a program on the basis of its evidence that 
it produces graduates who are competent, caring, and qualified educators, and that the 
program has the capacity to offer a quality program. The program presents this evidence in 
the Brief 

The whole point of the TEAC accreditation process is to test the claims that the program 
faculty makes in its Inquiry Briefer Inquiry Brief Proposal. TEAC verifies the evidence pre- 
sented in the Inquiry Brief and evaluates whether or not the evidence supports the pro- 
gram’s claim that it prepares competent, caring, and qualified educators. The quality of ev- 
idence and the quality of the system that produced it are the two key factors in the TEAC 
accreditation decision. In the case of the Inquiry Brief Proposal, TEAC verifies the evi- 
dence of the program’s capacity and plan to produce an Inquiry Brief \n\Vt\ evidence beyond 
what was required for state program approval. 

The Brief is, in essence, a research monograph (or, in the case of the Inquiry Brief Pro- 
posal, a plan for such a monograph), and should be focused on what the program faculty 
wants and needs to know about the program’s performance. It should run about 50 pages, 
and it should be based primarily on existing documents, such as reports of ongoing inquiry, 
other accreditation reports, and institutional research reports prepared for internal and ex- 
ternal audiences. 

The Inquiry Brief or Inquiry Sr/e/ Proposa/ should be meaningful to the program and con- 
tain information necessary to properly and responsibly administer and improve the pro- 
gram. It should be brief, and it should be about inquiry. Producing the Brief should be a 
seamless part of the program faculty’s normal, collective activity to improve the program. 

The program faculty members should work together to produce the Brief All faculty mem- 
bers of the program options represented in the Sr/e/ should contribute to the process, and 
TEAC requires that faculty members in the program review and approve the final Sr/e/ be- 
fore it is submitted for audit. 

TEAC reviews drafts of the Brief and works with the program faculty, providing feedback 
and guidance, until the Brief \s accepted for audit. 

The time it takes a program faculty to prepare a Sr/e/ varies, depending on local circum- 
stances, such a program structure, available documentation, state context, and the institu- 
tion’s commitment to the process. Generally, it takes the same amount of time as needed 
to produce a solid research article. 

To produce the Brief, TEAC recommends that program faculty follow the steps described 
below in the “Guidelines for producing the Brief.” In doing so, the faculty members will de- 
velop a comprehensive understanding of their program necessary to writing the Brief They 
will also be well-prepared for the audit. 
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Content of the Inquiry Brief and Inquiry Brief Proposal 

The Inquiry Brief 

To be accredited, an eligible program submits a research monograph, called an Inquiry 
Brief, in which the faculty and administrators present the evidence supporting their claims 
that their program satisfies TEAC’s three quality principles: 

1. Evidence of their students’ achievement and that their interpretation of their assess- 
ments of student achievement is valid, 

2. Evidence that the program monitors quality and systematically engages in continuous 
improvement that is based in part on information about its students’ learning, and 

3. Evidence of the program’s capacity for quality. 

Through the Inquiry Brief the program faculty members present qualitative and/or 
quantitative evidence that their graduates are competent, qualified, and caring and that the 
institution has the capacity to offer a quality program. 

The program faculty members document the evidence they possess about what their grad- 
uates have learned, the validity of their interpretations of the assessment of that learning, 
and the basis on which the program faculty makes its decisions to improve its program. To 
do this, the faculty members must show that they have a valid method for determining what 
their students have learned and accomplished. Then they must show that their students 
have learned the subject matter they will teach, the pedagogical subject matters of the field 
of education, and, most important, that their students can teach effectively and caringly. 

The faculty members must also show that they use what they learn about their students’ 
learning to improve both the program and the system they have in place for monitoring and 
ensuring the quality of the program. Finally, they must show that they have plans to 
undertake a systematic inquiry into the factors that affect the quality of the program and 
their students’ accomplishments. 

The Inquiry Sr/e/ focuses on what the program faculty wants and needs to know about the 
program’s performance. It includes the claims a faculty makes about its graduates’ 
knowledge and skill, a rationale for the assessments of those claims, the empirical basis of 
the validity of the evidence that is presented to support the claims, the findings related to 
the claims, and a discussion of what the evidence means and what has been learned from 
it. In addition, the Inquiry Brief reports on the faculty’s efforts to evaluate the rigor of its own 
quality control system and the adequacy of the program’s capacity to offer a quality pro- 
gram. 

Based primarily on existing documents, the Inquiry Brief contains only information and anal- 
ysis relevant to the case that the program prepares competent, caring, and qualified profes- 
sionals. 
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The Inquiry Brief Proposal 



Faculty members representing new programs, or programs that are in the process of col- 
lecting evidence for their claims beyond what they cite for state program approval, may 
submit for initial accreditation status an Inquiry Brief Proposal, in which they propose the 
method by which they will investigate the outcomes of their program and show that their 
graduates are competent, qualified, and caring, and that the program meets TEAC’s three 
quality principles. 

The Inquiry Brief Proposal is appropriate for new programs or programs that have been 
significantly revised in recent years. The program faculty members in these cases may not 
yet have sufficient recent evidence that meets a scholarly standards for their claims of 
student accomplishment, but they do have evidence in other areas. They have evidence of 
their capacity for program quality, evidence of a sound quality control system, evidence that 
the institution is committed to the program, and a plan for acquiring evidence over time to 
support their claims. 

The Inquiry Brief Proposal is a research proposal, a scholarly work like a grant or 
dissertation proposal, in which the program faculty members propose the method by which 
they will find evidence (qualitative, quantitative, or both) to demonstrate at a research 
standard level that the program’s graduates are competent, qualified, and caring. The 
program faculty members also present the evidence they used to achieve state program 
approval and demonstrate that they have a reasonable basis for thinking (1) that the 
program’s students have learned the subject matters they will teach; (2) that the students 
have solid pedagogical knowledge; and (3) that the students can teach effectively in a 
caring manner. In addition, the program faculty members provide a rationale for their 
assessments that explains why the faculty members think the assessments are reliable and 
valid and that the criterion for success for each is appropriate. 

Most importantly, the faculty members also show how they will use what they learn about 
their students’ learning to improve both the program and the system they have in place for 
monitoring and ensuring the quality of the program. In addition, they present their plans to 
undertake a systematic inquiry into the factors that affect the quality of the program and 
their students’ accomplishments. Finally, they also provide evidence that the institution has 
the capacity to offer a quality program. 

Like the Inquiry Brief, the Inquiry Brief Proposal is based primarily on existing documents, 
such as reports of ongoing inquiry, other accrediting and state review reports, and institu- 
tional research and publications. It contains only information and analysis relevant to the 
case that the program will be able to bring forward evidence that it prepares competent, car- 
ing, and qualified professionals. The Inquiry Brief Proposal \s also about 50 pages. 
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Guidelines for producing the Brief* 

TEAC recommends that program faculty follow these steps to produce the Brief. 



1. Review. Study and understand the TEAC process and requirements. Know the re- 
quirements for the three quality principles and the required components of the Brief 
Study the Guide to TEAC Accreditation and navigate the website ( www.teac.org) for 
the most up-to-date information. Also review state standards and protocols as ap- 
propriate, especially for those states with which TEAC has a partnership agreement. 
When in doubt, ask TEAC staff ( teac(5)teac.orq) . 

2. Gather information and prepare appendices. Invest time in examining the pro- 
gram thoroughly. We suggest that the program faculty gather and review all required 
information about the program, specifically — and in this order, from least to most 
complex — the information that will eventually appear in the program overview, the 
program requirements (Appendix D), the faculty qualifications (Appendix C), copies 
of any local instruments and rubrics used to assess candidate knowledge and skills 
(Appendix F), and the program’s capacity (Appendix B). It would be appropriate to 
assemble and draft these appendices as the second step. 

3. Inventory available measures. Continuing the examination of the program, the 
program faculty should examine the inventory of evidence in the field (Appendix E), 
noting what evidence the program relies on, what it does not, and what it might col- 
lect in the future. Once faculty has formulated claims, it may need to revisit Appendix 
E to identify evidence it considers irrelevant to its claims. 

4. Conduct an internal audit. Next, the program faculty should describe its quality 
control system, conduct an internal audit of the system, and draft an internal audit 
report (Appendix A). 

5. Take stock. TEAC suggests that the program faculty now meet together to review 
what they have learned about their program from steps 1-4. 

6. Formulate claims. Draft a set of statements that makes clear what the faculty be- 
lieves the program accomplishes with regard to TEAC's Quaiity Principie I (gradu- 
ates know their subject matter, have pedagogical knowledge, and have caring and 
effective teaching skills). These statements can be unique to the program and are of- 
ten aligned with state or national standards. They must be consistent with any other 
public statement of the program’s claims and be linked to the program’s assess- 
ments. 



* 



7. Draft the Brief. Analyze and interpret the results of the assessments identified in 
Appendix E, develop the case, and assemble a draft Brief Review the draft, using 
the two checklists (at the end of Tab 1) for programs and formative evaluators. Sub- 
mit a draft to TEAC. 



The word Brief is used to refer to both the Inquiry Brief and the Inquiry Brief Proposal. 
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Checklist for preparing the Brief 



1. Review 


Done! 


Date 


TEAC’s principles and requirements 






State & professional association standards 






TEAC’s accreditation process 






TEAC’s requirements for content of the Brief 






2. Gather information 


Done! 


Date 



Program overview 






Alignment of program requirements with Quality Principle / and 
state and national standards (Appendix D) 






Program faculty qualifications (Appendix C) 






Program capacity (Appendix B) 






Collect copies of all local assessments (Appendix F) 







3. inventory avaiiabie measures 


Done! 


Date 


Study the evidence available in the field pertaining to the graduates’ 
learning, note what evidence the faculty relies on currently, what it 
does not, and what it might collect in the future (Appendix E) 






Assemble a list of the program’s assessments and explain how and 
why the program uses them (rationale) 






Add any newly developed instruments to Appendix F 






Determine the program’s standard for the sufficiency of evidence of 
graduates’ learning that would support claims for Quality Principle 
(e.g., what are the cut scores?) 






4. Conduct an internai audit 


Done! 


Date 


Describe the program’s quality control system and conduct an 
internal audit 






Draft the internal audit report (Appendix A) 






5. Take stock 


Done! 


Date 


Review all materials and findings to date 






6. Formuiate ciaims 


Done! 


Date 


Write your claims and align claims with evidence for them 






Check consistency of your claims with your published public claims 






7. Draft Brief 


Done! 


Date 



Study the results of the assessments cited in Appendix E, and for- 
mulate the program’s interpretation of the meaning of the results of 
the assessments 






Compare draft against checklists for program and formative 
Evaluators 






Submit a draft to TEAC 
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TEAC resources 



TEAC offers the following print and electronic resources and guidance: 

Website. TEAC’s website (www.teac.org) is a comprehensive and up-to-date guide to 
TEAC, the accreditation process, and membership. Check the TEAC website regularly for 
updates on policies and procedures, announcements about events and members. TEAC 
members receive periodic email announcements. The website has a feature that encour- 
ages members and others to make suggestions and comments about TEAC. 

Publications 

Teacher Education Accreditation Councii. A short introduction to TEAC, including the ac- 
creditation process and principles and standards. ©2009 

Guide to Accreditation. A comprehensive guide for the faculty, staff, and administrators of 
TEAC member programs preparing for initial and continuing education. The guide includes 
a full description of TEAC’s principles and standards; the accreditation process, including 
the audit; and detailed instructions on writing the Brief. ©2010 [Two copies are sent to each 
program as a benefit of membership.] 

Guide to the TEAC Audit. A comprehensive guide to the audit process, including responsi- 
bilities of the program, TEAC staff, and auditors. Includes a checklist for tracking the audit 
process. ©2010 [One copy is sent to each program when the Brief \s declared auditable.] 

TEAC brochure. A brief, succinct description of TEAC and its quality principles, available on 
request to programs for use in informing campus leaders and others in advance of the audit. 
Three-panel brochure [size of a #10 business envelope] © 2009 

TEAC Operations Poiicy Manuai. A convenient reference for all TEAC members, staff, and 
auditors, board of directors, and members of the Accreditation Panel, this manual describes 
all of TEAC’s current policies and procedures. ©2010 

Guidance and feedback 

TEAC provides a staff liaison to assist the candidate programs while the faculty members 
are writing and editing the Brief offering feedback on methodological design, statistical 
analysis, and interpretations of evidence. TEAC funds this service from the audit fees all 
programs pay in the year of their audits. 

To further guide members in their process, TEAC offers a variety of additional formative 
evaluation services. Contact TEAC for the cost of these services: 

1. On request, TEAC can provide tailored workshops on-site for a program or group of 
programs (such as a sector- or state-based consortium). 

2. TEAC can provide individual consultation for program representatives in TEAC’s offices. 

3. In rare cases where a program requires or desires more help than workshops or the 
staff liaison can provide, TEAC can provide consultants on an individually negotiated 
cost basis. 
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Outline for a typical TEAC Brief 



1. Program overview 

Overall logic: guiding philosophy and orientation of 
the program 

Program areas, levels, specialties, and options 
Brief history of the program 
Program demographics 

Table of enrollment trends, numbers and types of 
students, numbers of faculty and types, etc. 

2. Claims and Rationale 

Statement of the claims (consistent with all relevant 
claims in the program’s literature) 

Reasons why the program thinks its assessments are 
valid and that the passing scores are appropriate 

3. Method of assessment 

Detailed description of the assessments 
Criteria for achievement or success 
Published information about the reliability and validity 
of the assessments 

Arguments for the content validity of the assessments 
Sampling procedure and procurement of evidence 

4. Results 

Results of the investigation into the reliability and 
validity of the assessments 
Evidence of stability and consistency of the measures 
Evidence of relationship, convergence, triangulation 
with other measures or evidence 
Results of the assessments with attention to the 
following issues: 

a. Significant digits 

b. Ranges of the scores and their variance 

c. Disaggregation of evidence 

d. Accurate and comprehensive table headings 

e. Sensitivity to insignificant differences 

f. Full disclosure of available evidence (all of the 
program’s cited evidence); See Appendix E. 

g. Evidence for each claim 

5. Discussion and Plan 
Discussion 

Meaning of the results: Were the claims supported? 
Were the results good news or bad news? 

Implications of the results for the program’s design 

Plan 

Steps to be taken based on the evidence: modifica- 
tions to the program, quality control system (QCS), 
plans for inquiry into the factors responsible for the 
results. 

6. References 

A list of any works cited in the Brief. 



7. Appendices 

Appendix A: Internal audit report 
Introduction: AudWors', faculty approval 
Description: Schematic and mechanisms of QCS 
Procedure: Audit plan and trail 
Findings: Discoveries about the QCS 
Conclusions: How well does QCS work? 

Discussion: Needed modifications in QCS or future 
audit procedures 

Appendix B: Capacity 

Evidence that the program is supported on a par with 
other programs at the institution 
Evidence that the program’s capacity is sufficient and 
adequate to satisfy 

Appendix C: Qualifications of the faculty 
Current academic rank and title 
Terminal degree, institution, field, and date 
Number of years of service 
Scholarly publications (number, type) 

Assigned courses in the program 
Awards, public school teaching, boards 

Appendix D: Program requirements 
Admissions requirements 
Course requirements and standards 
Course titles and descriptions 
Program standards and requirements 
Graduation requirements 
State license requirements 
Table of alignment of program option requirements 
with state and_national standards 

Appendix E: Full disclosure of all relevant and availa- 
ble evidence (including any evidence cited elsewhere 
in support of, or about, the program) 

a. Grades 

b. Standardized tests (entrance, exit, and license) 
about the graduates or the graduates’ own 
students 

c. Surveys of students, alumni, employers 

d. Ratings of portfolios, work samples, cases 

e. Basis for rates: hiring/promotion, certification, 
graduate study, professional awards, publica- 
tions, etc. 

f. Reasons for neglecting or rejecting certain catego 
ries of evidence 

g. Plan for inclusion of new categories of evidence 
in a subsequent Inquiry Brief 

Appendix F: copies of locally developed assessment 
instruments cited in the Brief 

Appendix G: status of educator programs accredited 
by other USDE or CHEA recognized accreditors 
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Organizing the Brief 

Within the TEAC system there is always considerable latitude in how the program faculty 
will make its case and what kinds of quantitative and/or qualitative evidence it will bring forth 
to support the case that it has fulfilled the requirements of TEAC’s system of accreditation. 

Most program faculty chooses to organize the Brief as a research article or monograph. 



Recommended article or monograph format 

1 . Program overview 

2. Claims and rationale for the assessments 

3. Method of assessing 

4. Results 

5. Discussion of results 

6. References 

7. Appendices 

A. Internal audit of quality control system 

B. Capacity for quality 

C. Qualifications of the program faculty 

D. Program requirements linked to relevant standards 

E. Inventory: disclosure of available measures used or declined 

F. Local assessment instruments 

G. Status of program options accredited by other recognized accreditors 



Alternate approaches 

As long as the Brief addresses all the elements, components, and subcomponents of the 

TEAC system (1.1 -3. 2.6), a program may take any approach that allows the faculty to best 

make its case. Some possible forms that may suite the faculty members are: 

1. An essay with sections corresponding to each of TEAC’s quality principles and stand- 
ards (1.1 -3.2.6); 

2. A comprehensive internal audit report that probes all dimensions of the TEAC quality 
principles (1 .1 -3.2.6); 

3. A full account of each numbered element, component, and subcomponent of the TEAC 
system (1.1 -3. 2. 6); 

4. A qualitative case study about their students’ accomplishments with regard to the quality 
principles and the program’s role in fostering them, controlling, and monitoring quality. 

5. A state or other report that aligns with each of TEAC’s requirements (1.1 -3.2.6) 

Each of these forms would be audited for accuracy and evaluated by the same processes 

as the recommended monograph format. 



TEAC believes that however the Brief is organized, writing the Brief shouid serve the 
program’s needs apart from TEAC accreditation. 
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TEAC’s evaluation of the Brief 



TEAC evaluates the Brief in a sequence of five steps, each one dependent on, and 
informed by, the ones before it: 

1 . formative evaluation (optional, see www.teac.org for a description TEAC’s services), 

2. auditability decision by the lead auditor, 

3. audit visit and audit report by the audit team, 

4. summative evaluation and recommendation by the Accreditation Panel, and 

5. accreditation decision by the Accreditation Committee. 

Each step is based on a set of questions. 

1. Formative evaluation 

Is the program making a persuasive case for itself? Does the Brief include all the required 
elements? Is the language clear and precise? 

The process of developing the Inquiry Brief or Inquiry Brief Proposal embodies the idea of 
continuous improvement. TEAC sees the Brief as a living document, so to speak, and 
consequently welcomes frequent consultation between the program faculty and TEAC 
about the Brief, particularly about effective approaches to substantiating the claims the 
program faculty makes. The TEAC staff sees its role as assisting the program faculty in 
making the best case possible that is consistent with the evidence the faculty has about its 
students’ accomplishments and related claims. For this reason, and at the program’s 
request, TEAC reviews draft Briefs and provides feedback and guidance and a range of 
services to those seeking accreditation. A key task of the TEAC staff’s formative evaluation 
of the Inquiry Brief \s checking the precision of the language and evidence. 

2. Auditability decision 

Is the Brief complete and ready to be audited? 

When the program faculty is satisfied with the case it has made, it submits a final draft of 
the Brief, complete with a covering checklist. TEAC staff completes a similar checklist that 
certifies that the Sr/e/ contains all the features required for an audit. This certification is a 
simple precaution and raises the probability that the audit will have a satisfactory outcome 
for the program and TEAC. Only then is the audit scheduled (or, if scheduled through a 
state protocol, confirmed). At that point the form of the Brief \s final and no changes, except 
minor editorial changes, are permitted. Any changes the program wishes to make after the 
Brief \s declared auditable and the audit has begun are made through the audit report find- 
ings. 

3. Audit 

Is the evidence in the Brief trustworthy? 

Through the audit, TEAC verifies the evidence the faculty cites in the Inquiry Brief or Inquiry 
Brief Proposal m support of its claims. 
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The auditors determine whether or not the evidence in the Brief \s trustworthy. To do so, the 
auditors need access to the raw data, spreadsheets, and documents upon which the 
authors of the Brief relied in writing the Brief The faculty should be prepared to show the 
TEAC auditors the data (records, journals, ratings, evaluations, transcripts, artifacts, etc.) 
that are portrayed in the Brief A simple rule is: if the authors needed to took at it, the 
auditors may aiso. Because the TEAC auditors will try to verify as much of the Brief as can 
be practically managed from the TEAC’s offices, the faculty may be asked to send the 
supporting source data to TEAC before the audit. By its very nature, a substantial portion of 
the audit, however, must be conducted on site. 

Audit of the Inquiry Brief. The main purpose of the audit of an Inquiry Brief \s to verify the 
evidence the program faculty have cited in support of its claims that the program meets 
TEAC’s three quality principles. The auditors select samples of evidence that they predict 
will reveal and represent the totality of the evidence the program faculty has presented in 
the Inquiry Brief The auditors are free to search for additional evidence in the process of 
the audit and these discoveries may support, strengthen, or weaken the corroboration of the 
evidence behind the program faculty’s claims with regard to the quality principles. 

Audit of the Inquiry Brief Proposal. The main purpose of the audit of an Inquiry Brief Pro- 
posal is to verify the targets associated with the program’s rationale, quality control system, 
commitment, and capacity, and to search on site for possible lines of evidence and ap- 
proach that can be used to support the program’s claims and the program’s method of es- 
tablishing reliability and validity of its evidence. The result is that a firm and realistic plan for 
the eventual Inquiry Brief can be established and negotiated between the auditors and pro- 
gram faculty. This entails verifying the statements the program faculty members have cited 
with respect to its rationale and the evidence they have for Quality Principles II and III. As 
with the audit of the Inquiry Brief, the auditors select samples of evidence that they predict 
will reveal and represent the totality of the evidence the program faculty have that their plan 
for an Inquiry Brief \n\W be successful. 

4. Summative evaluation 

Is the preponderance of the evidence in the Brief consistent with the program’s claims that 
its the program’s graduates are competent, caring, and qualified? Is the evidence reliable, 
valid, and of sufficient magnitude to support the program’s claims? 

TEAC’s Accreditation Panel determines if the evidence, as verified by the audit, is con- 
sistent with the program’s claims and the requirements of the TEAC system and also if the 
evidence is of sufficient magnitude to support the claims in the Brief On the basis of its 
examination and evaluation, the panelists recommend an accreditation status for the 
program to the Accreditation Committee. 

5. Accreditation decision. 

Should the Accreditation Panelists’ recommendation be accepted? Was the TEAC process 
that ended in the panel’s recommendation followed properly? 

TEAC’s Accreditation Committee makes two decisions: (1) whether TEAC followed its own 
guidelines and quality control system and (2) whether there is anything in the record that 
would call the panelists’ recommendation into question. In their deliberations, the panelists 
and the committee are guided by a set of common heuristics for the accreditation decision 
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but the panelists are satisfied if the preponderance of the evidence is consistent with the 
program’s claims. The committee assumes the panelists were correct and can only undo 
the panelists’ recommendation if there is conclusive evidence that the program’s claims 
were not true. 
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Who makes the decision within the TEAC System? 



Exercise 1 : For each conclusion that a TEAC review might reach, indicate whether the 
review would be made by the auditors (A), Accreditation Panel (P), Accreditation Committee 
(C), or none of the above (N). 





The evidence presented in Table 12 on page 10 of the Brief \s accurate. 




The evidence brought to bear on the claim of caring teaching is reliable. 




The program is accredited for five years. 




The Brief \s too short. 




The institution is committed to the program. 




TEAC followed correct procedures in handling this program’s case. 




It is recommended that the program be accredited. 




The Brief \s ready for audit. 




The Brief \s not in the proper format. 




The license test scores are too low to support the claim 1 . 




The program is awarded accreditation for two years. 




The license scores are 30 points lower than what was reported in the IB. 




There is a rival hypothesis for the program’s interpretation of its findings. 
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Understand the difference: the Inquiry Brief and Inquiry Brief Proposal 

Exercise 2 : Complete the table below by placing a check mark in the appropriate col- 
umns for each item. Some items will have a check mark in both columns, others in only one, 
and some may have none at all. Note that the task is about requirements. 



Comparing the Inquiiy Brief and Inquiry Brief Proposal 





Required in the 
Inquiry Brief 


Required in the 
Inquiry Brief Proposal 


1 . Claims related to Quality Principle 1 






2. Rationale linking assessments to claims 






3. Findings related to claims 






4. Description of how the program 
addresses cross-cutting themes 






5. Evidence of institution’s commitment to 
the program 






6. Evidence of program capacity to produce 
graduates who are qualified, competent, 
and caring 






7. US News ranking of the program 






8. Percentage of graduates employed 






9. Description of the methods used to col- 
lect evidence 






10. Explanation of the cut scores used to 
make decisions in the program 






1 1 . Evidence drawn from local assess- 
ments about the reliability of the measures 






12. Number of faculty with teaching license 






13. Conclusions related to the findings 






14. Self-audit of quality control system 






15. Appendix A (internal audit) 






16. List of decisions the faculty has made 
about the program based on evidence 






17. Appendix E (inventory of evidence) 






18. Pass rates reported for Title II 






19. Number of faculty and students 






20. GPA requirements for the degree 






21 . Description of quality control system 






22. Evidence of faculty endorsement 






23. Appendix F (local assessment instru- 
ments) 
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Exercise 3 : Monitor your progress in preparing an Inquiry Brief or Inquiry Brief Proposal. 

Checklist to accompany the submission of the inquiry Brief and inquiry Brief Proposai^ 

Requirements for the Brief Find it on Stiii missing 






1 ■ We identify the author(s) of the document. 

2. We provide evidence that the facuity approved the doc- 

ument. 

3. We give a brief account of the history and iogic of the 

program and its piace within the institution. 

4. We provide some demographics of program facuity and 

students (e.g., race and gender), broken out by year, by 
each program option. 

5. We state our ciaims expiicitiy and preciseiy. 

6. We provide evidence to support our ciaims organized by 

their reiationship to the components of QPi (1.1-1 .3). 

7. We provide evidence for aii the subcomponents of QPi 

(i.4): iearning how to iearn (1.4.1); muiticuiturai perspec- 
tives and accuracy (1 .4.2) and technoiogy (1 .4.3). 

8. We have checked that our ciaims are consistent with 

other program documents (e.g., cataiogs, websites, and 
brochures). 

9. in the rationaie, we expiain why we seiected our particu- 

iar measures and why we thought these measures wouid 
be reiiabie and vaiid indicators of our ciaims. 

10. in the rationaie, we aiso expiain why we think the crite- 

ria and standards we have seiected as indicating success 
are appropriate. 

11. We describe our method of acquiring our evidence - 

the overaii design of our approach, inciuding sampiing and 
comparison groups (if appiicabie). 

12. We provide at ieast two measures for each ciaim uniess 
there is a singie measure of certain or authentic vaiidity. 

13. For each measure we inciude empiricai evidence of the 

degree of reiiabiiity and vaiidity. 

14. We present findings reiated to each ciaim, and we offer 

a conciusion for each ciaim, expiaining how our evidence 
supports or does not support the ciaim. 

15. We describe how we have recentiy used evidence of 

student performance in making decisions to change and 
improve the program. 

16. We provide a pian for making future decisions concern- 
ing program improvements based on evidence of our stu- 
dents’ performance. 



' The checklist for the Inquiry Brief Proposal need not have entries for rows 6, 7, 13, 14 and 15. 
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17. We provide evidence that we have conducted an inter- 
nal audit of our quality control system (QCS) and we pre- 
sent and discuss the implications of the findings from our 
internal audit. 






18. We provide Appendix C that describes faculty qualifica- 
tions. 






19. We provide Appendix D that describes our program re- 
quirements and their alignment with state and national 
standards. 






20. We make a case for institutional commitment to the 
program (Appendix B). 






21 . We make a case that we have sufficient capacity to of- 
fer a quality program (Appendix B) 






22. We list all evidence (related to accreditation) available 
to the program (Appendix E). 






23. We provide copies of all locally developed assessments 
in Appendix F. 






24. We provide, if applicable, copies of decisions by other 
recognized accreditors for professional education programs 
not covered in the /nqu/ry 6r/e/ (Appendix G). 






25. If our program or any program option is delivered in dis- 
tance education format, we make the case that we have the 
capacity to ensure timely delivery of distance education and 
support services and to accommodate current student 
numbers and expected near-term growth in enrollment. 






26. If our program or any program option is delivered in dis- 
tance education format, we describe the process by which 
we verify the identity of students taking distance education 
courses. 
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Understanding a different style of writing 

Overview 

In what other ways are the Inquiry Brief and Inquiry Brief Proposal different from the self- 
study reports we have written in the past? 

This exercise explains TEAC’s expectation for precise language throughout the Inquiry Brief 
and Inquiry Brief Proposal. Below we provide a description of the kind of writing we seek 
from programs. To help you understand and practice this different style of writing, we offer 
two exercises (Exercises 4 and 5) on the pages immediately following. 

Precise ianguage 

Producing an Inquiry Brief or Inquiry 6r/e/Proposa/ calls for a kind of writing that is different 
from the usual self-study or program approval document. TEAC seeks a writing style that 
has greater precision and clarity than is typically called for in accreditation or state program 
approval self-study reports. 

Why does precise ianguage matter to TEAC? 

TEAC stresses clear and precise language because of the kinds of claims and supporting 
evidence that TEAC asks of its candidates for accreditation. 



Teacher education programs seeking TEAC accreditation must provide solid evidence that 
their candidates understand their subject matter and the process of learning and that they 
develop teaching skills. The programs must also demonstrate that they have an ongoing 
process for reviewing and improving themselves and the capacity to offer quality education. 
The program faculty’s claims and the measures used to support them are very specific; 
therefore the language must be precise. 



Vague, imprecise language will not only obscure the goals and accomplishments of the 
program, but it will make it more difficult for the auditors to verify the text of the Brief be- 
cause the auditors need to determine whether or not the errors they may find in the text al- 
ter the meaning of the Brief or would mislead a reader. Imprecise text is likely to be open to 
multiple interpretations, some of which may not even have been intended by the authors, 
but which if adopted may alter the meaning of the text and potentially mislead the reader. 
For this reason, checking the precision of the language and evidence of the Inquiry Brief 
and Inquiry Brief Proposal \s a key task in both TEAC’s formative evaluation and the audit of 
the Brief TEAC staff and auditors focus on language and precision in order to determine 
the degree to which the 6r/e/ means exactly what the program faculty intend it to say. 
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Understanding a different style of writing 



Exercise 4 : Consider the assertions below and ask yourself, How could one know this? 
Could that really be the case? What exactly does this mean? And the most important ques- 
tion of all - What evidence could make it wrong? 

1 . Our students acquire a deeply rooted matrix of the application of theory to practice. 

2. Students learn multiple ways of understanding education, with a balance of quantitative 
and qualitative methods. 

3. ...the program aims to develop reflective scholars who are expert sources of knowledge 
in their disciplines and who can tap the diversity of the education community at large to 
enrich their scholarly work. 

4. The core courses represent a coherent study of learners as they experience diverse con- 
texts for learning, the nature of the learning experiences and the substance of what is 
learned, as well as the larger policy environment in which schooling takes place. 

5. It is our view that weak enrollments here, and elsewhere, may be due to larger economic 
conditions, but it is more likely due to the fact that we are not successful in getting out 
[the message] that this is a world-class university with a world-class faculty, that we 
have an abundance of resources, and have a strong track record in successfully placing 
our graduate students. 

6. The faculty consciously spiral essential curricular elements into each course so that ex- 
posure to critical topics is recursive and affected by each intern’s experience in his/her 
own classroom and school. 

7. Classroom realities have a way of making a reasonable approach to a complex issue 
seem more pressing than abstract case studies. 

Exercise 4 , continued; Choose one sentence from the list above and describe what 
makes it problematic for TEAC’s purposes. Be prepared to discuss your thoughts with the 
group. 

For example, consider the assertion, “our students acquire a deeply rooted matrix of the 
application of theory to practice,” and ask: How could one know this? Could that really be 
the case? What exactly does this mean? And the most important question of all - What evi- 
dence could make it wrong? This kind of assertion is unacceptably imprecise and should be 
avoided in an Inquiry Brief because surely its author could not answer obvious questions 
about it - such how deeply rooted is the matrix, how one could tell if it were deeply or shal- 
lowly rooted, what there was before the matrix was acquired, what it is rooted to, what kind 
of matrix it is (orthogonal, oblique), how many cells the matrix has, how the theories con- 
nected to matrix, what specific practices flow from the theories, which others do not, wheth- 
er there are wrong-head applications of theory to practice, how many theories there are, 
and so forth. 
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Exercise 5 : Consider the following sentences that might appear in a Brief and rate the 
precision of each Check precise if the sentence seems sufficiently precise. Check not pre- 
cise if the sentence is seriously imprecise. Check ? if you are not sure or if the statements 
could be made more precise in subsequent text. 



^Note: an answer key for Exercise 5 appears at the end of this section. 



Statement 


Not precise 


Precise 


? 


1 . Our students leave the program with a caring attitude 
toward children. 








2. In leading a class discussion, our students exhibit 
knowledge of and a disposition to practice “wait time” in 
appropriate amounts. 








3. Graduates of our program are, overall, competent to take 
on the role of classroom teacher. 








4. Students in our program are proficient in the use of the 
tools of instruction, including PowerPoint, WebQuests, and 
word processing. 








5. Students in our program are taught “how to learn” in six 
different required courses. 








6. Our students receive a liberal education in our institution 
because of the distributional requirements set by the Facul- 
ty Senate for all students in the university. 








7. Students will graduate with passing scores on the state’s 
sub-test entitled, “Knowledge of Diversity. ” 








8. There is no difference in knowledge of subject matter be- 
tween our graduates in mathematics education and their 
counterparts who are pure mathematics majors. 








9. Our students will score at the 50^'^ percentile or above on 
the Minnesota Teacher Attitude Inventory administered af- 
ter the completion of the student teaching course. 








10. Our graduates are committed to the Judeo-Christian 
principles that made our nation great. 
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^Answer Key to Exercise 5, Understanding a Different Kind of Writing 

Imprecise statements usually will merit a probe by the auditor, so the test of an imprecise 
statement if often whether it requires a probe by the auditor to insure what the reader would 
understand the statement to mean or not mean. 



Note: “Precise” denotes that the sentence is an example of the writing preferred by TEAC. 



item 


Answer 


Expected auditor probe 


1 


Not precise 


The term “caring” is sufficiently ambiguous in the literature to 
prompt a probe here. What does the term mean, exactly? 
How is caring measured? What does the term, “attitude,” 
mean or what behavior represents it?. 


2 


Precise 


The term “wait-time” is well documented and precise. This 
use of language does not merit a probe; however, it may 
arise that the faculty use precise terms imprecisely, so an 
auditor might probe how the faculty knows the students fail 
to show wait time. 


3 


Not precise 


The term, “competent," has many meanings several probes 
are possible - Does it mean they survive the first year, or 
how competence was distinguished from incompetence or 
what are the subparts of being competent. 


4 


Precise 


The technical terms here are well understood and clear, alt- 
hough the meaning of the term, proficient, could be probed. 


5 


Not precise 


The phrase “how to iearn” has many meanings that would be 
probed. 


6 


Precise 


The assertion is in the form of a definition: This is what the 
faculty means by iiberai education. Although the definition 
may be problematic and contested, it is clear. 


7 


Precise 


Assuming the passing scores are public and known, this 
statement is clear. 


8 


Not precise 


The concept “knowiedge of subject matter is not fixed in the 
literature. This statement requires a probe before the reader 
could be confident in his/her interpretation of the assertion. 


9 


Precise 


This sentence is stated as a fact: it is clear what the faculty 
means. 


10 


Not precise 


The auditors would probably not find the statement relevant 
to the TEAC system but if it were, the meaning of “Judeo- 
Christian” and “committed” would be probed 



Exercise 6 : Appendix D: To represent program requirements and show how they are 
related to the state’s and professional association’s standards, complete the table on the 
next page for each program option. 
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Format for Appendix D for teacher education 



TEAC Quality Prin- 
ciple 1 


Program option requirements that address Quality Principle /and state subject 
matter and pedaqoqical standards for 


State 

standard 

number 


Professional 

association 

standard 

number 


components 


Required 

courses 


Field work re- 
quirements 


Admissions 

requirements 


Portfolio 

requirements 


Exit 

requirements 






1.1 Subject matter 
knowledge | 
















1.2 Pedagogical 
knowledge 
















1.3 Caring and 
effective teaching 
skill 1 
















1.4.1 Cross-cutting 
theme: Learning 
how to learn \ 
















1.4.2 Cross-cutting 
theme: Multicultural 
perspectives i 
















1.4.3 Cross-cutting ' 
theme: Technology 

















The program is free to tailor the column headings to its particular requirements for each of its program options. For example, a program might have made the fol- 
lowing provisions for subject matter knowledge in its program requirements for math teachers: the state may have some math standards the program names, the 
program may have adopted the subject matter standards of NCTM, certain math courses are required and named, some field work may require math lessons and 
units, for admission the program may require a math aptitude test score, some prerequisite math courses, a portfolio may require work samples of math lessons 
and their assessment by the student teacher, and finally the program may require some exit survey on math preparation and knowledge or some standardized 
math test (e.g.. Praxis II). All of these requirements would be cited in the cells in the table above either directly and/or by reference to some other document. A 
program may have requirements of a different kind from those in the column headings above to address Quality Principle I, etc. and these should be cited either by 
substitution or addition. 
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Format for Appendix D for educational leadership 



TEAC Quality Prin- 
ciple 1 


Program option requirements that address Quality Principle /and state subject 
matter and pedaqoqical standards for 


State 

standard 

number 


Professional 

association 

standard 

number 


components 


Required 

courses 


Field work re- 
quirements 


Admissions 

requirements 


Portfolio 

requirements 


Exit 

requirements 






1.1 Professional 
knowledge 
















1 .2 Strategic deci- 
sion-making 
















1.3 Caring and 
effective leadership 
skill 1 
















1.4.1 Cross-cutting 
theme: Learning 
how to learn \ 
















1.4.2 Cross-cutting 
theme: Multicultural 
perspectives i 
















1.4.3 Cross-cutting ' 
theme: Technology 

















The program is free to tailor the column headings to its particular requirements for each of its program options. For example, a program might have made the fol- 
lowing provisions for subject matter knowledge in its program requirements for math teachers: the state may have some math standards the program names, the 
program may have adopted the subject matter standards of NCTM, certain math courses are required and named, some field work may require math lessons and 
units, for admission the program may require a math aptitude test score, some prerequisite math courses, a portfolio may require work samples of math lessons 
and their assessment by the student teacher, and finally the program may require some exit survey on math preparation and knowledge or some standardized 
math test (e.g.. Praxis II). All of these requirements would be cited in the cells in the table above either directly and/or by reference to some other document. A 
program may have requirements of a different kind from those in the column headings above to address Quality Principle I, etc. and these should be cited either by 
substitution or addition. 
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Teacher Education Accreditation Councii (TEAC) 



EXERCISES 

Exercise 7 : Commitment: is your institution committed to your teacher education pro- 
gram? On a scale of 1-10 (with 10 indicating full commitment), how committed is your insti- 
tution to your program? How do you know? List some signs that prompt you to give the rat- 
ing you did. Discuss your ratings and reasons with your colleagues. 

123456789 10 



Exercise 8 : Commitment: Predict where your program’s statistics would be superior, 
the same, or inferior to the norm at your institution. Working with a colleague, write Equal to, 
Higher than, or Lower than in each cell of the last column of the table below. 



Capacity dimension 


Program 


institutionai norm 
for similar programs 


Analysis of 
differences 


3.1.1 Curriculum^ 








3.1.2 Faculty^ 








3.1.3 Facilities'^ 








3.1.4 Fiscal and 
administrative^ 








3.1.5 Student support 
services® 








3.1.6 Student 
feedback^ 









^ E.g., total number of credits for graduation, grade requirements, number of credits in the major 
^ E.g., proportions of terminal degrees, genders, races, etc., research support per faculty member, work- 
load composition, balance of academic ranks, promotion and tenure standards 
E.g., allocated space and equipment, support facilities, special facilities 
^ E.g., cost/student, staff compensation, scholarships, proportion of administrators/support staff 
® E.g., counseling, advisement, media/tech support, career placement 
E.g., proportion of complaints about program, course evaluation ratings 
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Teacher Education Accreditation Councii (TEAC) 



Exercise 8 helps you begin to make your argument for your institution’s commitment to your 
program. The expectation is that a program has parity with others in the institution. If the 
program is treated differently, you will need to think about why that is the case. 



Exercise 9 : Commitment: Assume you have verified the evidence for each of the 
statements below. Now consider whether you would want to probe further any of the follow- 
ing findings. Do they indicate commitment, or are there alternative interpretations you can 
think of that do not signify commitment and that you want to check with additional internal 
audit tasks? Circle the numbers of those findings for which you think there may be other in- 
terpretations that require additional audit probes on your part. Be ready to defend your se- 
lections. 

1 . The program has a higher proportion of part-time faculty than the institution as a whole. 

2. The program has a higher proportion of assistant professors than the institution as a 
whole. 

3. The program charges less tuition for its teacher education graduate program than for 
other graduate programs in the institution charge. 

4. The program is one of three programs in the strategic plan adopted by the trustees. 

5. The average length of service of the program faculty is less than the average of faculty 
in other programs within the institution. 

6. Students in the program file fewer complaints per capita than students in other pro- 
grams. 

7. The program receives the same per faculty budget allotment as other programs. 

8. The students in the program rate their courses higher on average than students in other 
programs rate their courses on average. 

9. The program has more secretarial staff than other similar departments. 

10. More credits are required in our program than any other undergraduate program. 



©TEAC ♦ One Dupont Circle ♦ Suite 320 ♦ Washington, DC ♦ 20036 ♦ 202/466-7236 ♦www.teac.org* 

26 




Teacher Education Accreditation Councii (TEAC) 



Exercise 10 : Commitment: Consider those instances where you entered a “lower 
than” rating for the program in Exercise 8 above. 

There may be a perfectly sound reason, unrelated to commitment, for your rating. Your pro- 
gram, for example, may have proportionally fewer full professors than the institution as 
whole, but the reason may be that a large number of retired full professors have been re- 
placed by new assistant professors. Or your program may have proportionally more part- 
time faculty than the institution as a whole. This may be because there is truly a lack of 
commitment to the program because your institution is seeking to minimize its costs, or it 
may be because there are sound reasons why the quality of the program would be en- 
hanced by proportionately more faculty members with real world an6 pracf/ca/ knowledge. 

Circle any “lower than” rating in Exercise 8 that can be explained as indicating something 
other than a lack of commitment and use the space below for your notes on these items. 
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Teacher Education Accreditation Councii (TEAC) 



Exercise 1 1 : if you found the following, what argument would you make and what addi- 
tional information would you seek in your inquiry? Write your response below. 

The education department average salary for 2007-08 was $39,453 and the average 
for faculty in the college whose service (2 years) equaled the college’s faculty’s ser- 
vice was $42,690. The difference was explained on the basis that some recent hires 
in other departments were a minority faculty member and faculty in computer sci- 
ence and in mathematics, all hires affected by competitive market factors. 



Exercise 12 : Parity. With your colleagues, select one capacity dimension (curriculum, 
faculty, facilities, resources, fiscal and administrative, support services, policies and practic- 
es) and outline a plan for presenting evidence to show that your program conforms to the 
norms for similar programs at your institution. Be prepared to present your plan to the whole 
group. 
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Teacher Education Accreditation Councii (TEAC) 



Appendices E and F 
Examine assessments 

At this point, it wiii be usefui to examine in depth aii the assessments that the program has 
avaiiabie: 

• Which ones provide evidence that the program prepares competent, caring, and quaii- 
fied educators and which ones do not? 

• Which ones specificaiiy provide evidence that the students know the subject(s) they wiii 
teach, know pedagogy, and know how to teach effectiveiy in a caring way and which 
ones do not provide that evidence? 

The facuity members shouid take the time to expiain why they think it is reasonabie to 
use the particuiar measures of student iearning they have seiected. The facuity members 
may beiieve that such measures as grades in the major courses, scores on Praxis ii, scores 
on the state curricuium tests, scores on the GRE subject matter test, grades on the senior 
thesis in the major, cogency of the candidates’ iesson pians in their subjects aii suggest that 
the teacher education candidates know their subject matter, if they beiieve this, then they 
wouid aiso expect that these measures wouid be reiated to each other. Students who score 
weii on one shouid score weii on the others and vice versa. These expectations shouid aiso 
be checked. 

The program facuity shouid aiso examine how the assessments are administered. The in- 
formation coiiected at this time shouid inciude instructions given to the assessors and any 
training that is provided to them that is designed to increase the reiiabiiity and vaiidity of the 
assessments. Having this information wiii heip the program facuity assembie a fuii picture of 
its assessments. Later, during the audit, the auditors wiii need it, too. They wiii inspect the 
assessment instruments, the instructions to the assessors, and any training materiais asso- 
ciated with the assessments. Aiso, you wiii need a copy for each of your iocai assessments 
for Appendix F, so this is a good time to create that appendix. 

Exercise 13 : Measures that will yield evidence: Beiow is a iist of hypotheticai 
measures taken on candidates in a program, indicate the coiumn in which each couid be 
inciuded in the tabie for Appendix E (on the next two pages), or whether the measure is 
even appropriate to use as evidence: 

1 . Mid-term exams given in Professor Noien’s methods ciass 

2. SAT scores of candidates 

3. Rate of students who are from out-of-state 

4. Grades in a technoiogy ciass 

5. “Pop” quizzes given in the educationai psychoiogy course 

6. Ratings assigned to candidates in student teaching by university supervisors 

7. Seif ratings by candidates of their abiiity to work with diverse students 

8. Rate of graduates who become nationaiiy board certified 

9. Grade point averages of candidates in eiectives 

10. GRE subject matter test resuits 

1 1 . Drop-out rate from the program 

12. Pass rates on iicense tests 

13. Program students’ mean iQ scores 
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Inventory: status of evidence from measures and indicators for TEAC Quality Principle 1 


Type of evidence 


Available and in the Brief 


Not available and Not in the Brief 


Note: items under each category are exam- 
ples. Program may have more or different 
evidence 


Relied on 

Reasons for including the 
results in the Brief 

Location in Brief 


Not relied on 

Reasons for not relying on 
this evidence 

Location in Brief 


For future use 

Reasons for including in 
future Briefs 


Not for future use 

Reasons for not including in 
future Briefs 


Grades 


1 .Student grades and grade point averages 












Scores on standardized tests 


2. Student scores on standardized license or 
board examinations 












3. Student scores on undergraduate and/or 
graduate admission tests of subject matter 
knowledge and aptitude 












4. Standardized scores and gains of the program 
graduates’ own pupils 












Ratings 


5. Ratings of portfolios of academic and clinical 
accomplishments 












6. Third-party rating of program’s students 












7. Ratings of in-service, clinical, and PDS teach- 
ing 












8. Ratings, by cooperating teacher and college / 
university supervisors, of practice teachers’ work 
samples. 












Rates 



® Assessment results related to TEAC Quality Principle I that the program faculty uses elsewhere must be Included in the Brief. Evidence that is reported to the 
institution or state licensing authorities, or alluded to in publications, websites, catalogs, and the like must be included in the Brief Therefore, Title 11 results, grades 
(if they are used for graduation, transfer, admission), admission test results (if they are used), hiring rates (if they are reported elsewhere) would all be included in 
the Brief 
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Inventory: status of evidence from measures and indicators for TEAC Quality Principle 1 


Type of evidence 


Available and in the Brief 


Not available and Not in the Brief 


Note: items under each category are exam- 
ples. Program may have more or different 
evidence 


Relied on 

Reasons for Including the 
results In the Brief 

Location in Brief 


Not relied on 

Reasons for not relying on 
this evidence 

Location in Brief 


For future use 

Reasons for including in 
future Briefs 


Not for future use 

Reasons for not including in 
future Briefs 


9. Rates of completion of courses and program 












10. Graduates’ career retention rates 












1 1 . Graduates’ job placement rates 












12. Rates of graduates’ professional 
advanced study 












13. Rates of graduates’ leadership roles 












14. Rates of graduates’ professional 
service activities 












Case studies and alumni competence 


15. Evaluations of graduates by their own pupils 












16. Alumni self-assessment of their 
Accomplishments 












17. Third-party professional recognition of grad- 
uates (e.g., NBPTS) 












18. Employers’ evaluations of the 
program’s graduates 












19. Graduates’ authoring of textbooks, 
curriculum materials, etc. 












20. Case studies of graduates’ own pupils’ 
learning and accomplishment 
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Exercise 14 (for teacher preparation) How the evidence listed in Appen- 
dix E aligns to the TEAC system: With a colleague, complete the table below, not- 
ing the assessments the program uses to measure subject matter knowledge, pedagogical 
knowledge, and caring and effective teaching skills, and making the case that the assess- 
ments for each outcome are valid. 



Developing a rationale for assessments, part 1 



Outcome 

Our graduates 
meet the. . . 


Measures 


Evidence that the measures are 
valid 


Subject matter 

knowledge 

claim 






Pedagogical 

knowledge 

claim 






Caring and 
effective teach- 
ing skills claim 






Cross-cutting 
theme of learn- 
ing how to 
learn 






Cross-cutting 
theme of multi- 
cultural per- 
spectives and 
accuracy 






Cross-cutting 
theme of 
technology 
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Exercise 14 (for educational leadership) How the evidence listed in Ap- 
pendix E aligns to the TEAC system: With a colleague, complete the table below, 
noting the assessments the program uses to measure professional knowledge, strategic 
decision-making, and caring and effective leadership skills, and making the case that as- 
sessments for each outcome are valid. 



Developing a rationale for assessments, part 1 



Outcome 

Our graduates 
meet the. . . 


Measures 


Evidence that the measures are 
valid 


Professional 

knowledge 

claim 






Strategic 
decision- 
making claim 






Caring and 
effective 
leadership 
skills claim 






Cross-cutting 
theme of learn- 
ing how to 
learn 






Cross-cutting 
theme of multi- 
cultural per- 
spectives and 
accuracy 






Cross-cutting 
theme of 
technology 
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Exercise 15 (for teacher education) Identifying the program’s standard: 

With a colleague, complete the table below, noting the assessments the program uses to 
measure subject matter knowledge, pedagogical knowledge, and caring and effective 
teaching skills, and cite the program’s standard for success on the measure, how was the 
standard determined and why is it appropriate. (See Exercise 41 , pages 62-63.) 



Developing a rationale for assessments, part 2 



Outcome 

Our graduates 
meet the. . . 


Measures 


What is the program’s standard for 
success on the measure and why is 
it appropriate? 


Subject matter 

knowledge 

claim 






Pedagogical 

knowledge 

claim 






Caring and 
effective teach- 
ing skills claim 






Cross-cutting 
theme of learn- 
ing how to 
learn 






Cross-cutting 
theme of multi- 
cultural per- 
spectives and 
accuracy 






Cross-cutting 
theme of 
Technology 
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Exercise 15 (for educational leadership) Identifying the program’s 
standard: With a colleague, complete the table below, noting the assessments the pro- 
gram uses to measure professional knowledge, strategic decision-making, and caring and 
effective leadership skills, and cite the program’s standard for success on the measure, how 
was the standard determined and why is it appropriate. (See Exercise 41 , pages 62-63.) 



Developing a rationale for assessments, part 2 



Outcome 

Our graduates 
meet the. . . 


Measures 


What is the program’s standard for 
success on the measure and why is 
it appropriate? 


Professional 

knowledge 

claim 






Strategic 
decision- 
making claim 






Caring and 
effective 
leadership 
skills claim 






Cross-cutting 
theme of learn- 
ing how to 
learn 






Cross-cutting 
theme of multi- 
cultural per- 
spectives and 
accuracy 






Cross-cutting 
theme of 
technology 
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Exercise 16 : Below are listed some “local instruments” that might be used by the faculty 
and placed in Appendix F. Circle the number(s) of the options that would likely be inappro- 
priate for inclusion in the Brief. 

1. A 10 minute quiz administered in a required Educational Psychology course in the 
program. 

2. The rubrics that cooperating teachers use to effect their evaluations of student 
teachers. 

3. A description of the State test required of all candidates seeking state licensure. 

4. A survey administered to graduates as they exit the program. 

5. A prompt for writing lesson plans that is a standard program assignment across all 
methods courses. 

6. Observation forms used by University/College supervisors during the time they su- 
pervise student teachers. 

List some examples of local instruments you might include in Appendix F not cited in the 
first six options. 



Exercise 17 : TEAC asks for the faculty’s reasons for including a measure in the Brief as 
support for a claim. Which of the following options represent reasonable, sensible reasons 
for including a measure? Explain your selections. 

1 . The state requires the measure. 

2. We’ve always used this measure in our program. 

3. It’s available to us - and easy to collect and to summarize. 

4. The measure was developed by faculty who studied precisely how it is sensitive to 
our claim. 

5. The faculty studied the in-house research carried out by ETS to validate the meas- 
ure and its suggested cut scores, and found the evidence provided to them compel- 
ling. 

6. We believe the measure is valid. 

7. The faculty undertook task analyses to demonstrate to its satisfaction that the meas- 
ure “gets at” the sub-component under study. 

8. This measure correlates nicely with other measures of this construct, giving the fac- 
ulty confidence that it can be relied upon as a measure of this sub-component. 

9. The university requires all programs to use its assessment of writing. 

10. We have found over the years that the students who scored above 85 on the meas- 
ure tended to become “Teachers of the Year” in the state. 
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Exercise 18 : TEAC asks for the faculty’s reasons for not including a measure in the Brief. 
Which of the following options represent compelling and sensible reasons for excluding a 
measure from the 6r/e/ proper? Explain your selections. (Remember that all measures hav- 
ing to do with candidate learning or institutional learning that are available to the program 
have to be made available to TEAC, if not in the Brief then in some other supplementary 
report). 

1 . We cannot think of an argument that connects the measure to the quality of our pro- 
gram. 

2. We don’t have this measure in SPSS format, so it would be difficult for us to analyze it or 
to summarize it. 

3. When we analyzed the measure, it didn’t support our claims so we omitted it. 

4. It is not cost-effective to collect this data year after year. It is available to us now - but 
only because it was collected for some other purpose. 

5. Our analysis of the reliability of this measure demonstrated that the measure cannot be 
trusted. Reliability coefficients were below .60. 



©TEAC ♦ One Dupont Circle ♦ Suite 320 ♦ Washington, DC ♦ 20036 ♦ 202/466-7236 ♦www.teac.org* 

37 




Teacher Education Accreditation Council (TEAC) 



Appendix A: 

The internal audit 

Exercise 19 : Using TEAC’s system (elements 1.0 through 3.0, components and sub- 
components) note which parts of your program’s quality control system yield information 

about student learning, valid assessment of student learning, institutional learning, and ca- 
pacity for quality. 

Ask yourselves the following questions: 

a) What procedures or mechanisms does your faculty rely upon to make sure faculty mem- 
bers in the program are competent? 

b) What procedures does your faculty rely upon to make sure the program’s courses are 
current, rigorous, and aligned with program goals? 

c) What procedures or mechanisms does your faculty rely upon to make sure the students 
are capable of meeting the high standards of the program and are likely to graduate on 
time? 

d) What procedures or mechanisms does your faculty rely upon to make sure that the 
teaching in the program is of high quality? 

e) What procedures or mechanisms does your faculty rely upon to make sure that the 
classrooms in which courses are delivered are appropriately equipped? 

f) What procedures inform the faculty about candidates’ performance on measures sensi- 
tive to TEAC’s quality principles and cross-cutting themes? 

g) What procedures provide faculty with evidence about program quality that prompts con- 
sideration of program changes and improvements? 

h) What procedures or mechanisms does your faculty rely on to insure that most students 
complete the program and secure a professional position? 

I) What procedures or mechanisms does your faculty rely on to insure that the faculty de- 
velop professionally and stay current in their fields? 
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Exercise 20 : Elements of the Quality Control System. Using the space below, 
sketch out the principal components of the quality control system relevant to your program. 
Focus on the quality associated with students, faculty, curriculum, field experiences, advis- 
ing, and whatever you deem important. Consider how the procedures of the quality control 
system improve the program. Be prepared to share your work with the group. 

You’ll find it helpful to make a list of all the pieces of the quality control system first. Then 
arrange them to reflect their purposes and relationships to each other. 
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Exercise 21 : Follow an audit trail. The program internal audit comprises a series of 
audit tasks. Each task is made up of a target and a probe. In the figure below, the “check 
on” arrows represent the probes and the ovals represent the targets. The topic areas are 
represented by the diamonds. This audit trail begins with a student folder and the figure in- 
dicates what it led to. Follow the trail and add at least four tasks and probes. 
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Exercise 22 : The following statements are examples of ones that might be included in 
your Brief, check the space 0 in front of the statements that would be a useful target to 
probe in your internal audit. Mark the space with an X if the statement that would not be 
useful to investigate. 





1 . Our graduates are well received by the field, especially by the principals. 




2. Our institution was founded in 1889. 




3. Most of our students are admitted into our program straight out of high 
school. 




4. Our program is recognized as representing high quality by the US News and 
World Report. 




5. Ninety percent of those we admit finish our program in eight semesters. 




6. Our graduates are reflective practitioners. 




7. We have a dedicated faculty. 




8. Our dean was appointed in 1993 after spending two years as Associate Dean 
here. 




9. Our university president is C. Byron Fipps. 




10. The mean Praxis 1 math score for our graduates was 182. 




1 1 . The proportion of assistant professors in the program is the same as it is in 
the college overall. 




12. Our program ranks first in the state in the Title II report. 




13. All our adjunct professors are screened for appointment by the same proce- 
dures we use in hiring for tenure track positions. 




14. Forty percent of the pupils of our graduates score in the proficient range on 
the NAEP examinations in social studies. 




15. The inter-rater agreement on our student teaching ratings is 88%. 




16. All of our students are church members and claim they are devout. 




17. Our courses are understood to be more rigorous than the courses in other 
programs. 




18. Seventy-five percent of graduates have teaching positions by the September 
following their graduation from the program. 




19. The correlation between our students’ grade indices between fall and spring 
semesters is .90. 




20. The annual report of the trustees shows that our program has been desig- 
nated as one of three flagship and signature programs of the college. 




21 . Most of the state’s “teachers of the year” are graduates of our program. 




22. Our department is one of six of 30 departments at the institution to have its 
own departmental secretary. 




23. Most of our students do local volunteer work with children. 



Exercise 23 : On the next page is an internal audit plan used by a TEAC-accredited pro- 
gram. Examine its features. 
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Plan for the internal audit (Number of students, etc., from sample in each category (yes, no, NA) 



Yes No NA 


Audit Questions 


Comments 




Students 

1. Did students meet admission requirements? 

a. Undergraduate GPA of 3.0 or higher 

b. Undergraduate major/concentration in approved field (pre-service only) 

c. GRE scores of 1000 -i- 4.5 (or 1500) or higher 

d. Two reference letters 

e. Personal statement 

f. Provisional certification (in-service only) 

g. Positive recommendation from Arts & Sciences (MAT only) 

h. Positive recommendation from Education 

i. Exceptions to a-h justified and documented 






























































2. Did students reflect program efforts to admit and retain- 

a. Diverse students with demonstrated potential as teachers? 

b. Teachers for high demand areas? 

c. Teachers for high need schools 




























3. Did any students change program areas after admission? 

If so, was the transfer documented and a new plan of study developed? 










4. Did any students transfer any credit? 

If so, was the transfer credit policy followed? 










5. Did any students take any courses on-line? 

If so, were the on-line course guidelines followed? 










6. Did students have plan of study that reflected program/degree 
requirements? 










7. Did students meet requirements for program retention? 
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8. Did students have adequate options for fieldwork affiliated with 
coursework and Practicum? 










9. Did students meet requirements for admission to Practicum? 






10. Did students complete program and degree requirements? 

a. Courses specified in plan of study or program description 

b. Graduate GPA of 3.0 or higher 

c. Project (Childhood, Inclusive Education) or 
Portfolio (Adolescence, Literacy, Special Education) 




























11. Were students recommended for certification in the area of study? 










12. Did students complete the degree in an appropriate timeframe? 
















Student Support 

13. Was each student assigned an advisor? 

If yes, did the advisor provide timely and useful advising? 

14. Did students seek assistance from office staff? 

15. Did students receive any GA/TA support from the program or 
campus? 

If yes, was it comparable to funding available to other students on 
the campus? 

16. Did students use any of the campus support services? 

17. Did students file any complaints with the Division of Education 
or School of Education and Human Development? 

If yes, were written procedures to deal with student complaints 
followed? 






















































Program and Courses (30% of Education courses randomly selected from 
transcript) 

18. Were all Education courses approved by the Division of 
Education? 

19. Were all Education courses approved by the Graduate School? 

20. Were required Education courses part of a program approved by 
NYSED? 

21. Were required Education courses listed as such in the Graduate 
School Bulletin and SEHD website? 
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Faculty (for randomly selected courses above) 

22. Were all or most courses taught by faculty members in tenure- 
track positions? 

a. Was the faculty member hired pursuant to a national search? 

b. Did he/she have a doctorate in a field related to the course con- 
tent? 

c. Was he/she reviewed periodically by faculty and administrators 
prior to promotion and tenure? 

d. Was he/she promoted and tenured within seven years of being 
hired? 

e. Was he/she reviewed periodically by faculty and administrators 
after promotion and tenure? 














































23. Were other courses taught by adjunct faculty members with a 
Master’s degree and relevant experience? 

24. Were courses evaluated by students using (a) SOOTs and/or 
(b) instructor-designed instruments? 

25. Did faculty use student feedback to improve courses? 


















Facilities, Equipment, and Supplies 

26. Were courses in classrooms of appropriate size with adequate 
seating for class? 

27. Were courses held in classrooms with suitable equipment and 
supplies? 

28. Were courses held in classrooms with adequate lighting, heat, and 
ventilation? 
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Exercise 24 : Design an audit plan: in the space below design an audit plan. Start 
by identifying the point at which you will enter the system (and why you are choosing this 
point). Then list the categories of questions. Sketch out a possible sequence of the audit 
(what leads to what). Be prepared to share your plan with the group. 

In a financial audit, auditors often sample a collection of cancelled checks. They then ask 
standard questions about each check: who authorized the purchase, where is the item pur- 
chased, and so forth. Alternatively, they might begin with a random sample of deposit slips, 
and ask, where is the invoice that corresponds to this deposit, who authorized the invoice, 
where is the money that was deposited, and so forth. Using financial audits as an analogy, 
consider ways of sampling your program’s quality control system. Where could the internal 
auditors begin? Suggest several ways. 
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Exercise 25 : After your internal audit, how would you answer any of the following ques- 
tions about your quality control system? 

1 . How were cut scores for various assessments arrived at, and are they succeeding in 
assuring student success in the program? 

2. Are there criteria for identifying excellence as well as for identifying minimally ac- 
ceptable competence? What benefits are (or could be) derived from identifying out- 
standing students? 

3. Do our faculty members understand our Quality Control System? 

4. How are the elements of the Quality Control System related to each other? 

5. How does the functioning of the Quality Control System improve student learning? 

6. In what ways does or could the Quality Control System inform faculty efforts to en- 
sure and/or enhance program quality? 

7. Is the program system of monitoring quality working well? 

8. How does the program compare to other programs in the State? 

9. How many courses make use of technology? 

10. Do the mechanisms we designed to select outstanding students actually find out- 
standing students? 

11 . Do the mechanisms we designed to select and retain outstanding faculty members 
actually find and retain outstanding faculty members? 

12. Do any of our current efforts to monitor quality actually have the opposite effect and 
impede progress and improvement? 
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Internal audit results 
EXERCISES 

Exercise 26 : Suppose your program requires the Myers-Briggs test and you found the 
following. What would be your conclusion about your quality control system? 

A search of recent graduate and undergraduate program completers was completed. 
Of the seven completers, four did not have Myers-Briggs scores recorded. The fol- 
low-up probes with the chair revealed that the faculty advisor is responsible for en- 
suring the completeness of the advisees’ files. It is possible that the Career Devel- 
opment Center, which administers the tests, may not have forwarded the scores to 
the Education Department. The Associate Director of the Center reported that the 
Career Development Center only forwards copies to those students who designate 
education as their field of choice and after students have given their permission and 
when requested (by phone or email) by the Education Department. It was not possi- 
ble to verify whether the four students had actually taken the tests. 



Exercise 27 : If your program had a requirement that students needed to earn a grade of 
C or better for each outcome evaluated in ED4030 and, after looking at six student folders, 
you found one of the following outcomes; what would your conclusion be with regard to 
whether your QCS was working with regard to this area with regard to each outcome? In the 
final row of the table, enter your conclusion for each of the three outcomes - are you follow- 
ing your policy confirmed or not? 



Three possible outcomes for the grades on the transcripts of six students ED4030 



STUDENT 


OUTCOME 1 


OUTCOME 2 


OUTCOME 3 


1 


C+ 


C 


A 


2 


c- 


I 


A 


3 


B 


D 


Course Absent 


4 


B 


A 


C 


5 


A 


A 


F 


6 


A- 


A 


A 


Audit conclusion 
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Exercise 28 : in your efforts to verify your policy of a C in ED4030, would you sample 
more students for any of the three outcomes cited in Exercise 27 above? If so, how many 
more would you need investigate for the results found in each outcome? 

Outcome 1 

Outcome 2 

Outcome 3 



Exercise 29 : Addressing your audit findings; Team up with three other work- 
shop participants and imagine that you are members of a group that has just completed an 
internal academic audit. How would you deal with the following findings? 

a. What if you found, as this faculty did, that many of the program’s students saw only ad- 
junct or part-time faculty instructors? 



Table 29a 

Number of hours in the major and education, GPA, and the percentages of 
the students’ courses taught by full-time and adjunct faculty for a random 
sample of twelve program students 



Student 


Major 


Hours 

Major 


Hours 

educ 


GPA 


FT 

faculty 


Adjunct 

Faculty 


1.CHD1 


Science 


32 


39 


2.99 


66% 


33% 


2.CHD2 


NA 


39 


39 


4.00 


0 


100% 


3.AES1 


Social Stud 


48 


30 


3.81 


0 


100% 


4.AES2 


Social Stud 


18 


24 


3.80 


0 


100% 


5.AES3 


Social Stud 


48 


30 


2.67 


0 


100% 


6.AES4 


Social Stud 


18 


24 


3.59 


0 


100% 


7.ECL1 


Social Sc 


32 


36 


3.61 


66% 


33% 


8.ECL2 


NA 


39 


39 


3.95 


66% 


33% 


9.AEE1 


English 


18 


24 


3.67 


66% 


33% 


10.AEE2 


English 


18 


24 


3.60 


0 


100% 


11.AER1 


Earth Sc 


46-49 


30 


3.83 


100% 


0 


12.AER2 


Earth Sc 


18 


24 


3.60 


0 


100% 
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b. What if you found, as did the faculty in the program represented in the table below, that 
there were significant differences in the ways faculty and cooperating teachers evaluated 
the students in the program? 



Table 29b 

Mean GPAs, MoStep I, overall teacher and faculty ratings for students 
in elementary and secondary teacher education program [sample N=15] 



Measure 


Elementary N=10 


Secondary N=5 


P values 


GPA 


3.47 


3.54 


.77 


A&S GPA 


3.71 


3.70 


.37 


ED GPA 


3.71 


3.70 


.37 


MoStep I 


3.80 


3.80 


1.00 


Cooperating 


38.60 


33.00 


.004 


Supervisor 


36.30 


37.40 


.60 
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c. What if you found that the correlations between the clinical part of the program and the 
rest of the program indicators were like those in the table below (actual data)? 

Table 29c 

Correlations among the clinical measures and grades and SAT sco res 



N=170 




Hi 


GPA 
3 sem 




Math 

251 


SAT 

Sum 


SATv 


SATm 


Clinical 


PGI (hs+) 




.62* 


.63* 


.62* 


.36* 


.67* 


.60* 


.54* 


.02 


GPA spring 






.96* 


.84* 


.53* 


.48* 


.45* 


.36* 


.16 


GPA 3 sem 








.89* 


.54* 


.47* 


.36* 


.45* 


.16 


GPA final 










.57* 


.41* 


.57* 


.37* 


.26* 


Math 251 












.37* 


.19* 


.43* 


.14 


SAT sum 














.83* 


.87* 


-.07 


SATv 
















.45* 


-.06 


SATm 


















-.06 



* significant at the .01 level or lower 
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Expanded information required for continuing accreditation 

The program’s first Inquiry Sr/e/ contained three implicit promises for the future and these 
need to be addressed in the program’s subsequent bid for reaccreditation. These three 
were: a plan to undertake continuing inquiry into the factors that might influence candidate 
learning and accomplishment in the program; evidence that not only did the Quality Control 
System work more or less as it was designed, but that it improved program quality; and that 
some of the categories of evidence, cited in Appendix E, that were not available or relied 
upon in the first Inquiry Brief \nou\6 be used in the subsequent Brief 

The Inquiry Brief irom program faculty seeking continuing accreditation will make the case 
for accreditation with TEAC by including all the familiar elements outlined on pages 41-69, 
but within that framework, the program will also need to integrate information about the 
three points above into its Inquiry Brief ior re-accreditation: 

1 . With regard to its plan for future and on-going inquiry, the faculty can provide a separate 
report of how the plan turned out, or the report can be included in the Inquiry Brief \r\ the 
Discussion section if it does not exist in a separate format. The program is not obligated 
to conduct the inquiry it planned in its first Brief, but it is obligated to have conducted 
some inquiry to earn a full continuing accreditation term. If the program abandoned its 
initial plan for inquiry, it would simply give the reasons for its going in a different direction 
and report the results of the inquiry it in fact undertook. 

2. With regard to evidence that the activities of the Quality Control System actually im- 
proved something in the program, the faculty should report the evidence it has that it 
has made something better in the program. This evidence may be the same as that un- 
dertaken in Item 1 above or it may be in some other area of interest to the faculty. 

3. With regard to how the evidence promised in Appendix E “for future use” has been ad- 
dressed, the faculty may either include it or provide reasons for not using it. 

There is always the hope and expectation that the faculty seeking reaccreditation will also 
have refined and enhanced the quality of the evidence it uses to make its case so that it is 
more persuasive and conclusive than what was submitted in the prior Inquiry Brief 



Exercise 30 : Following up on the findings (for re-accreditation): With the 
key question of the internal academic audit in mind {Did the mechanism make things better, 
improve quality?), consider the case below: 

CASE: The faculty members in a teacher education program were concerned that their ad- 
mission standards, while functioning as designed, were not yielding the outcomes they 
should. 

In their internal audit in their first Inquiry Brief, they had examined the relationship between 
GRE scores and program grade point averages and found the correlation was too low to 
justify the continued use of the GRE as a predictive measure of success in the program. 
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Subsequently, they then undertook an examination of a sample of students who had com- 
pleted the program and who had the various undergraduate indices reported in the table 
below. 

What conclusions should they come to in their second Inquiry Brief about how should they 
use the undergraduate index, if at all, in their admission procedure in their graduate pro- 
gram? 



Table 30 

Number of master’s students above and below at graduate GPA of 3.5 



as a function of various undergraduate grad 


le point indices (2.4, 2.5, 2.75, 3.0) 


Undergraduate GPA 


Program GPA 
Below 3.5 


Program GPA 
Above 3.5 


Total 


Above 3.0 


8 


74 


82 


Below 3.0 


9 


28 


37 










Above 2.75 


11 


87 


98 


Below 2.75 


6 


15 


21 










Above 2.5 


13 


96 


109 


Below 2.5 


4 


6 


10 










Above 2.4 


14 


99 


113 


Below 2.4 


3 


3 


6 


Totals 


17 


102 


119 



©TEAC ♦ One Dupont Circle, Suite 320 ♦ Washington, DC 20036 ♦202-466-7236 ♦ www.teac.org 

52 




Teacher Education Accreditation Council (TEAC) 



Claims 

EXERCISES 

Exercise 31 : Study the claims below. Are they appropriate to the TEAC framework? 



Claim 


Acceptable 


Perhaps 

acceptable 


Not 

acceptable 


1 . Our graduates know their subject 
matter. 








2. Our graduates score above the state 
average on the Praxis II test. 








3. Our graduates have a deep understand- 
ing of the Christian philosophy that forms 
the basis of this religious school. 








4. Our graduates almost always know the 
answers to any question about content 
asked by their pupils. 








5. Upon graduation, our students will have 
a deep and abiding understanding of the 
subject matter they are prepared to teach. 








6. Because our graduates all major in the 
subject matters they intend to teach, they 
are well prepared as teachers in subject 
matter knowledge. 








7. We recommend for certification only 
those persons among our program com- 
pleters who score above the state cut- 
score on the state subject matter tests. 








8. Based on ratings provided by university 
supervisors and by cooperating teachers, 
our graduates know their subject matter. 








9. Our graduates are familiar with ways of 
using technology to locate information 
about subject matter pertinent to planning 
and implementing lessons. 
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Exercise 32 : Borrowing claims: Discuss with your colleagues how you might pre- 
sent your claims in the Brief. Will you use state or national frameworks, or develop your own 
set of statements? If you use “borrowed claims” or standards from another source (such as 
INTASC or ISLLC), be sure you understand the standard (for example, if you claim INTASC 
Principle #1, The teacher understands the central concepts, tools of inquiry, and structures 
of the discipline(s) he or she teaches and can create learning experiences that make these 
aspects of subject matter meaningful for students, the faculty should be clear what the “es- 
sential concepts and tools of a discipline” are). 

Recall the discussion in Exercise 4 about precise language and how it would apply to such 
a claim. An auditor would surely ask for examples of central concepts and tools of inquiry 
and what some of the structures might be in the discipline(s). 

Use this space to cite the evidence you might use to support INTASC Principle #1 . 
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Exercise 33 : Writing claims: With your colleagues, use the pages below to draft a 
set of claims, aligning them with each of the three components of TEAC’s Quality Principle 
/: student learning in the areas of subject matter, pedagogical knowledge, and caring and 
effective teaching skills for teacher preparation, and student learning in the areas of profes- 
sional knowledge, strategic decision-making, and caring leadership skills for educational 
leadership. When drafting your claims, keep in mind the three cross-cutting liberal arts 
themes of learning how to learn, multicultural perspectives and accuracy, and technology, 
and how these themes are incorporated into the components of Quality Principle I. 

If you are using state or national standards, map those standards against the three compo- 
nents of Quality Principle I. 

1.1 Subject matter knowledge (or Professional knowledge) 
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1.2 Pedagogical knowledge (or Strategic decision-making) 
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1.3 Caring and effective teaching skiiis (or Caring and effective ieadership skiiis) 
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Exercise 34 : Making claims consistent with the evidence you have: Turn 
back to the inventory that the faculty has prepared for Appendix E of the Brief {page 24). 
Does your program have the evidence necessary to substantiate the claims it makes to the 
public and to TEAC? 

Return to the claims you created in Exercise 33 and for each claim, list the evidence you 
would need to support it, noting what is currently available or accessible. 



Exercise 35 : Being consistent with public claims: Check your claims about the 
program’s outcomes against the statements you make to the public via websites, brochures, 
catalogs, mission statements, state program approval reports, and so forth. 

Reflect on the statements about student outcomes that your program is currently making in 
its public literature. What student learning claims do you and your colleagues currently 
make concerning your graduates? Are these claims appropriate for the goals and mission of 
the program and for the evidence upon which you rely? Are they consistent with the claims 
you are making to TEAC? Do you have evidence to support the public claims? 



Exercise 36 : Grain size: For the following claims, judge whether or not the claim rep- 
resents a sufficient grain size for accreditation purposes. Regardless of grain size issues, 
identify other problems with the claim, if any. 

1 . Our graduates have mastered Bloom’s Taxonomy. 

2. Our graduates are well received in the field. 

3. Our graduates are excellent teachers. 

4. Our graduates are seen to be excellent teachers. 

5. Our graduates receive average ratings above 3.5 on a 5 point scale. 

6. All of our graduates have an academic major. 

7. Our graduates successfully complete a technology course. 

8. Our graduates reflect upon their teaching. 

9. University supervisors all have recent K-12 experience. 



©TEAC ♦ One Dupont Circle, Suite 320 ♦ Washington, DC 20036 ♦202-466-7236 ♦ www.teac.org 

58 




Teacher Education Accreditation Council (TEAC) 



Exercise 37 : Connect your claims and assessments: Using the claims that you 
and your colleagues have developed, complete the form below to examine what (and why) 
you rely on to assess student learning, why you think the measure is valid, and why the cri- 
teria and standards you have selected as indicating success are appropriate. 



Claim: 

Source of Evidence: 

1. Is this the only evidence you have for this claim or are there other sources that will 
be used? 

This is the only source of evidence for this claim. 

There are other sources of evidence for this claim. 

Specify: 



2. How will the evidence be reduced or scored so that it might be used to bear on the 
claim? 

Scores will be generated by using a scoring key 

Check list 

Content analysis 

Scoring rubric 

Results from a testing service or the state 

Data from transcripts or other documents 

Other 



3. Will there be a cut score or a standard XhaX will enable you to interpret the find- 
ings? 

Yes No Working on it 



4. How will you address the reliability issue for this measure? 



5. How will you address the validity issue concerning your interpretation of this 
measure? 



Repeat this exercise for each measure relevant to the claim. 
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Exercise 38 : Begin developing plans to test the credibility of your particular claims. With 
your colleagues, select a claim about student learning your faculty members might make 
about the program. Design a plan that includes the following: 

a) Two measures (at least) taken over a representative period 

b) Procedures for assessing the reliability of the measures 

c) Procedures for making a persuasive case for the validity of the interpretations the 
faculty will be making of the measures 

d) Option: If you plan to sample, include sampling procedures and descriptions of 
procedures for testing the fit between the sample and its population 



Exercise 39 : Draft your rationale. Using the space below, draft a short argument for 
one of your assessments for one of your claims that shows 

1. Why the assessment was selected? (The answer to this question often entails showing 
how the assessment procedures reflect the features of the program, e.g., graduation re- 
quirements, admission criteria and procedures, coursework, field assignments, and ex- 
periences.) 

2. What the passing score is for the assessment, how it was determined, and why does the 
faculty think it is appropriate? 

3. What is the faculty’s basis for thinking the assessment is reliable and that they can in- 
terpret the results validly? 

You might produce the draft together with colleagues from your campus or write individual 
drafts and then compare and combine drafts. 
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Exercise 40 : The program’s standards: An expectation for the rationale section is 
to argue for the cut or passing scores that are in use in the program. When a measure is 
given, what level does the faculty take to be acceptable? Whatever the answer to that ques- 
tion, TEAC asks that the rationale address the question: What makes this faculty judgment 
credible? 

Some examples of cut scores include: How high does the GPA have to be to graduate from 
the program? How high does the score on the state licensure test have to be to be judged 
by the faculty as “competent”? What levels of ratings from cooperating teachers are ex- 
pected before a candidate is dropped from the program? What are the stated expectations 
for advancing to student teaching in the program’s screening process? 

a. List the current cut scores that apply to your program: 



Assessment 


Cut score 


SAT 

GRE 

High school index 
License scores 
GPA 

Selected course standards 
Satisfaction survey ratings 
Student course evaluations 
Course 
Instructor 

Entry to student teaching 
Education GPA 
Subject Matter GPA 
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b. Below are some proposed arguments for particular cut scores. Which arguments do you 
find compelling? Which ones do you find less than convincing? Mark those that are compel- 
ling with a C in front of its number. Mark those that are not convincing with an N. Place a 
question mark in front of those for which you are unsure. 



C, 

N, 

or ? 


Argument for cut scores 




Prior to entering the TEAC process, there were a number of cut scores in place. We 
intend to examine each of them as we develop habits of “institutional learning.” As 
of now, we have chosen to leave most of them in place until our inquiry can chal- 
lenge them and suggest changes in them. 




We wrote to the programs^ that were honored by Carnegie in the Teachers for a 
New Era project to find out what cut scores they used in three particular areas - 
GPA to exit the program; GPA to enter student teaching; and satisfactory levels of 
ratings received from cooperating teachers on a five point scale. We adopted their 
cut scores. 




We hired a statistician who has worked with the National Board for Professional 
Teaching Standards (NBPTS) to help us empirically set cut scores. She worked for 
over a year with our files that included a complete data set for 500 of our graduates 
over a five year period. Her procedures yielded cut scores for GPA to exit, GPA to 
enter student teaching, and the lowest level satisfactory rating score received from 
cooperating teachers. We adopted our consultant’s recommendations. 




We couldn’t find in the research literature or in our own findings guidance for setting 
our cut scores. For this reason, we adopted the TEAC suggested 75% guideline. All 
of our cut scores represent the application of the 75% rule to our empirical maxi- 
mum score. 




In a faculty meeting, we discussed the issue of cut scores and based the discussion 
on files of students who were near the current cut scores. After lengthy give and 
take among faculty members, we voted to approve the cut scores reported in the 
Brief. 



® www.teachersforanewera.com 
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Exercise 41 : Figuring out the passing score (The Angoff method) 

The faculty has used the following form of eight items to rate a student teacher’s proficiency 
with technology. The rating form allows the rater to score each item as “satisfactory” or “un- 
satisfactory.” Using this form, the highest possible score is 8. The faculty asks: What should 
represent a cut score on this rating form to distinguish students who meet the department 
expectations from those who don’t meet the department expectations? To determine the cut 
score, the faculty used the Angoff method (Livingston & Zieky, 2004). Here is the proce- 
dure: 

1. Identify judges who know the students in the program and who are familiar with their 
practices with technology in the classroom. (For this exercise, you and your colleagues 
at the workshop will be judges). 

2. Think about a borderline student in your program - one who falls near the cut between 
competent in technology and incompetent in technology. Discuss this student in some 
detail with your colleagues or in a “conversation with self” if you are the only person from 
the program in the workshop. Describe his practices in the classroom and his uses of 
technology in some depth. 

3. Work to understand the following steps of the procedure: For each of the 8 items, decide 
how likely it would be for borderline students, such as the one you selected and de- 
scribed in step 2 above, to be rated satisfactory on this item. Score each item as 1 if the 
borderline student would know it and 0 if not. Determine the total score correct for each 
rater and take the mean of those scores as the cut-score (rounded up or down). 

4. Practice the Angoff method on the scale we have provided. What cut score do you rec- 
ommend? 

Directions'. Rate the borderline student 1 if he or she could do the item or 0 if he or she 
could not do the items for each of the 8 items of this scale. 



1 orO 


Rating scale for the use of technology in the classroom 




1 . The student teacher (intern) uses Power Point (or another presentation ap- 
plication) when presenting information to the class. 




2. The student teacher (intern) uses the Internet to connect the class with oth- 
ers doing similar work in schools across the nation and/or the world. 




3. The student teacher (intern) uses the Internet to locate sources appropriate 
for the students to use. 




4. The student teacher (intern) finds software items that are useful for teaching 
understanding of difficult concepts. 




5. The student teacher (intern) coaches students to use computers for many 
purposes. 




6. The student teacher (intern) is careful to preview software for appropriate- 
ness and efficacy before it is introduced to the class. 




7. The student teacher (intern) shows ingenuity in selecting software for use in 
the classroom. 




8. The student teacher (intern) uses well software to help manage the class, 
record grades, register feedback, and/or to communicate to students and their 
parents. 



Reference: Livingston, S.A., & Zieky, M. J. (2004). Excerpts from passing scores. Princeton, NJ: Educa- 
tional Testing Service. 
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Another way to think about it, if there were 100 students who were judged borderline in 
technology, how many of the 100 would likely be rated satisfactory on each of the 8 items. 
When you are finished with this task, each item should have a number assigned to it from 0 
to 100. Sum the numbers you have assigned to the 8 items, and divide by 100. The quotient 
should estimate the cut score on the rating scale. For example, if the estimates for the 8 
items were as follows: 40, 20, 30, 20, 40, 10, 5, 10 - the sum is 175 and the quotient is 
1.75. Rounding to the nearest integer, we have determined that the cut score should be 2. 
Any student receiving a 2 or lower on the scale should be deemed not to have met faculty 
expectations. 



Exercise 42 : The rationale requirement in both the Inquiry Brief and the Inquiry Brief 

Proposal asks that an argument be advanced to justify the selection of assessments. Which 

of the following arguments might be used in the rationale section? Circle the number(s) of 

the arguments that might be used in a rationale. 

1. A description of the process by which the instrument (survey, rubric, test, interview 
schedule, etc.) was developed. 

2. A description of a critique the instrument received from outside reviewers. 

3. A content analysis showing how the instrument addresses elements of the construct that 
is being assessed - caring teaching, subject matter knowledge, etc. 

4. If the assessment is a standardized measure published by a testing company, simply 
say so in the argument. 

5. Let TEAC know that if the auditors are interested in the validity of the instrument, they 
should write to the publisher. 

6. Simply advance the assumption that since the local authors of the instrument are all well 
prepared for their roles in the program, they must have developed a valid instrument. 

7. Carry out a pilot study with candidates in the program, and assess the reliability of the 
instrument empirically and report the estimated reliability coefficient. 

8. Report the program’s historical experiences with the assessment instrument. 

9. Explain how evaluators were trained in the use of the assessment. 



Exercise 43 : Evidence of reliability: Which of the following approaches would yield 
evidence that the faculty would find compelling about the reliability of the evidence for the 
claim, “our students know how to apply technology in the classroom”? Circle the number(s) 
of the approaches that your faculty would find credible. 

1. For a 10 item rating form completed by methods instructors, a coefficient alpha is pro- 
vided, with a value of .82. 

2. The faculty observes that the means of a 10 item rating form completed by methods in- 
structors across four sections of the course are almost identical. 

3. Two methods instructors rate a sample of students in the program independently, and 
the level of agreement between the ratings is perceived to be high. 

4. The level of agreement of the two methods instructors cited in option 3 above is as- 
sessed with a correlation coefficient - and is found to be .85. 

List other evidence that would convince the faculty that the measures were reliable. 
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Exercise 44 : Validity: The faculty is interested in knowing whether the 10-item scale 
used to assess the program’s claim concerning technology was valid as a useful tool to veri- 
fy the claim. Circle the number(s) of the approaches for assessing validity that your faculty 
would find credible. 

1 . Since the measures were found to be reliable, the issue of validity is no longer relevant. 
If the measures are reliable, they are surely valid. 

2. The students’ scores on the ten-item scale on technology are correlated with the ratings 
they received in student teaching on “uses technology effectively.” The correlation be- 
tween these two measures is .75. 

3. The faculty reviewed the ten items on the technology scale and determined that the 
items covered all of their intentions about what students should learn about technology 
in their program. The scale was judged to have content validity. 

4. The ratings on the scale discriminated between those students who used technology 
well in student teaching and those who did not - a finding yielded by a discriminate 
analysis of the evidence. 

List other approaches that would yield evidence that the faculty would find compelling about 
the validity of the ten-item scale. 



Exercise 45 : Measures truly relied on: Review the following novel and idiosyncrat- 
ic measures uncovered in TEAC audits and consider the evidence upon which the program 
faculty truly rely: 

• Candidates equal or exceed majors in grades in the disciplines (teaching subjects) 

• Faculty noted the exceptionality of those as students who later were board certified 

• High faculty agreement in rating quality of random samples of students by name only 

• A&S departments hire candidates as graduate teaching assistants (GTAs) 

• Local superintendents waive interviews for recommended students 

• Higher state scores in schools with higher densities of program graduates 

• Candidates are the first choice and accept their first choice in employment 

• Candidates are first choice of cooperating teachers for student teaching assignments 

• Lawful patterns of correlations among internal and external measures of the availa- 
ble measures of competence 

• Work samples with student/pupil learning data 

• Authentic artifacts (viz., technology, video) 

• Comparisons of retention of program’s students in teaching with other programs 

• Regents or NAEP examination scores for candidates 

• Reporting assessments at various stages in the program to show reductions in vari- 
ance over time 

• On-demand ratings by faculty of students, video-taped lessons show lawful correla- 
tions with internal & external measures 

• Pupil evaluations of student teachers 
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Exercise 46 : Organizing your data: With your colleagues, try organizing a spread- 
sheet like the one below for a sample of your students. Fill in the column headings for as 
many data sources as you have. 

Each row contains the data for one and only one unique student in your sample. Each col- 
umn contains something you know about your students that is important to the quality of 
your program. (Example data sources are provided below.) The cells in the spreadsheet 
contain information (qualitative and/or quantitative) about each student. 



Student characteristics 



Student 


Year 


Option 


Level 


Gender 


Race 


Major 


Site 


Etc. 


1. 


















2. 


















3. 


















N 



















Admissions indicators 



Student 


SAT 

score 


ACT 

score 


Rank in 
H.S. 


H.S. 

Grades 


Interview 


Writing 

Sample 


Etc. 


1. 
















2. 
















3. 
















N 

















Grade point indices 



Student 


GPA in 
methods 


GPA in 
major 


GPA in 
clinical 


Grades 
in techn. 


Grades 
in MC 


Etc. 




1. 
















2. 
















3. 
















N 

















Locai program measures and ratings 



Student 


Field ex- 
perience 


Coop. 

teacher 

rating 


College 

supervisor 

rating 


Portfolio 

artifacts 


Self- 

ratings 


Etc. 




1. 
















2. 
















3. 
















N 

















License tests and other externai measures 



Student 


Praxis I 


Praxis II 


GRE 


Etc 








1. 
















2. 
















3. 
















N 
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Post-graduate and employer surveys (and tl 


he like) 


Student 


Rating 
of prgm. 


Rating of 
courses 


Rating of 
faculty 


Years 

teaching 


Employer 

Rating 


Pupil state 
tests 


Etc. 


1. 
















2. 
















3. 
















N 

















Exercise 47 : Results: The results, whether quantitative or qualitative, should be truly 
representative of the program under review and not idiosyncratic to a particular time period 
or circumstance. 

The results must also be disaggregated by subcategory when an aggregated presentation 
would mask important differences within the groups and categories being reported. 

In cases where a program is undergoing revisions and renewal, the results should be of a 
character that will support a sound prediction of what future results will be. Generally, this 
means that the most recent results will carry greater weight in TEAC’s decision making. 

The exercises that follow ask you to think about ways you might present results in the Brief 
and what pitfalls you can avoid. 

The four tables below describe some hypothetical and real findings related to the claim of 
subject matter knowledge {Quality Principle 1.1) for a program that prepares secondary lev- 
el mathematics teachers. Consider the following tables reporting findings in support of the 
claim Our graduates know their subject matter. 

Are there problems with this presentation that are serious enough for you to reject the con- 
clusion that the program satisfies Quality Principle I with regard to subject matter 
knowledge? How could the array be made clearer? Use the space below each table for 
notes in preparation for the conversation. 
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Table 47a 

Findings related to the claim of subject matter knowledge for a program 



to prepare secondary level matl 


hematics teachers 




GPA in upper level mathematics 
courses: math department mean: 
3.1 


Scores on Praxis II: mathematics 
national mean: 540 


Year of 
graduation 


Graduates 

N 


Mean 


SD 


Graduates 

N 


Mean 


SD 


06-07 


15 


3.5 


.5 


15 


610 


90 


07-08 


12 


3.4 


.4 


12 


590 


95 


08-09 


14 


3.6 


.5 


14 


615 


92 


09-10 


20 


3.0 


1.1 


20 


510 


130 


10-11 


15 


3.5 


.5 


15 


610 


95 



Note: The average correlation across the five-year period between the two measures was .75. The faculty as- 
sessed the reliability of the GPA by drawing a sample of 30 students from the five-year period, and correlating 
the grades received on the odd lines on their transcript with the grades received on the even lines. The correla- 
tion was .78, providing support for a satisfactory level of reliability. ETS reports that the Praxis II examination in 
mathematics has a reliability of .83 for its norming group. The graduates of our program match well the charac- 
teristics of the ETS norm group. 
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An improved presentation of the data appears below. 
J^ote: you will find a sample response to this table on page 69. 



Table 47b 

Mean (and SD) GPA and Praxis II Scores and Pass Rates in Secondary Mathematics 
for Years 2006-2011 at Exemplar University 





GPA in upper level mathematics 
courses: math department mean: 
3.1 ; math department standard: 
2.75 


Scores on Praxis II: mathematics 
national mean: 540 
State cut score: 520 


Correla- 
tion: 
GPA- 
Praxis II 


Year of 
graduation 


Number gradu- 
ates/ 

number admitted 


Mean 

(0-4) 


SD 


Graduates 

N 


Mean and 
(pass rates) 
400-800 


SD 


Pearson 

R 


06-07 


15/20 


3.5 


.5 


15 


610 (84%) 


90 


.89 


07-08 


12/21 


3.4 


.4 


12 


590 (77%) 


95 


.70 


08-09 


14/22 


3.6 


.5 


14 


615 (85%) 


92 


.69 


09-10 


20/22 


3.0 


1.1 


20 


510 (47%) 


130 


.71 


10-11 


15/20 


3.5 


.5 


15 


610 (84%) 


95 


80 


Total 


76/105 


3.4 


.6 


75 


587 (75%) 


100 


.75 



Note: The faculty assessed the reliability of the GPA by drawing a sample of 30 students from the five-year 
period and calculating the mean GPA each year. The means were within .05 of each other. ETS reports that 
the Praxis II examination in mathematics has a reliability of .83 for its norming group. The graduates of our pro- 
gram match well the scores and demographics of the ETS norm group. 



The table below offers an example of how a program faculty might organize its quantitative 
results for the components of Quality Principle I. 

Note that although means and standard deviations are the likely entries in each cell of the 
table, frequency counts, ranks, percentages, percentiles, or whatever quantitative metric the 
faculty relies on could be also entered. 
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Table 47c 

Means (standard deviations) of a sample of 80 students in six categories 
of assessments in support of claims for Quality Principle I 



Outcome claims* 


Categories of evidence and range of scores 


The program’s 
graduates have 
acquired. . . 


Grade 

point 

index 


Standardized 

tests 


Faculty & 
Cooperating 
Teacher 
evaluations 


Student 

self- 

reports 


Survey of 
graduates 
and employ- 
ers 


Gains in 
pupil 

scores on 
work 
samples 




(Score 
Range, 
e.g 1-4 
& Cut 
Score) 


Praxis 

(Score 
Range & 
Cut 
Score) 


NES 

(Score 
Range 
& Cut 
Score) 


Fac 

(Score 
Range 
& Cut 
Score) 


Coop 

(Score 
Range 
& Cut 
Score) 


(Score 
Range & 
Cut Score) 


Grad 

(Score 
Range 
& Cut 
Score) 


Emp 

(Score 
Range 
& Cut 
Score) 


(Score Range 
& Cut Score) 


Subject matter 

(Professional 

knowledge) 




















Pedagogy 
(Strategic 
decision -making) 




















Teaching skill 
(Leadership skill) 





















*lncludes measures of learning to learn, multicultural perspectives and technology 
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Here is another example: 



Table 47d 

Mean grades (u/g & program), license tests, portfolio rating, 
internship ratings (mid-term and final) and course pedagogy projects for 
students in special education program options 





N 


Minimum 


Maximum 


Mean 


Program 

standard 


Standard 

deviation 


Undergrad (1-4) 


43 


2.60 


3.88 


3.29 


3.00 


.32 


ProgGPA (1-4) 


43 


3.33 


4.00 


3.84 


3.00 


.14 


LAST (100-300) 


15 


233 


286 


262 


250* 


18.45 


ATS (100-300) 


15 


234 


284 


265 


250* 


14.74 


LIT (100-300) 


30 


223 


289 


265 


250* 


16.11 


SWD (100-300) 


35 


222 


279 


254 


250* 


14.18 


CST (100-300) 


23 


202 


283 


259 


250* 


19.77 


Portfolio 1 (1-3) 


43 


1.00 


3.00 


2.36 


2.25 


.52 


Portfolio 2 (1-3) 


43 


1.00 


3.00 


2.29 


2.25 


.68 


Portfolio 3 (1-3) 


43 


1.00 


3.00 


2.45 


2.25 


.60 


Portfolio 4 (1-3) 


43 


1.00 


3.00 


2.00 


2.25 


.72 


Domain 1 (1-4) 


43 


1.80 


4.00 


3.69 


3.00 


.42 


Domain 2 (1-4) 


43 


2.50 


4.00 


3.73 


3.00 


.39 


Domain 3 (1-4) 


43 


2.00 


4.00 


3.69 


3.00 


.40 


Domain 4 (1-4) 


43 


1.80 


4.00 


3.81 


3.00 


.39 


Mid Term (1-4) 


42 


2.30 


3.70 


2.92 


3.00 


.35 


Final (1-4) 


42 


2.50 


4.00 


3.32 


3.00 


.35 


Project 1 (1 -1 00) 


32 


80.00 


97.00 


90.92 


75.00 


4.52 


Project 2 (1-4) 


32 


3.30 


4.00 


3.77 


3.00 


.27 



*State passing score is 220 
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^ Sample response: 

Looking at the data in Tabie 47b, you might suggest to the hypotheticai Brief authors that 
they investigate their record for the foiiowing: 

• The statistics about the vaiidity of the interpretations of the measures. 

• Whether the reiiabiiity sampie of 30 was representative of the popuiation and wheth- 
er the A/'s are representative sampies of the graduates or the universe of graduates. 

• The range of scores for Praxis ii (what is the zero score and the maximum?). 

• The compietion rates in each year of the program (because in the years when fewer 
than 20 students graduated, the program might have had high drop-out rates, in 
which case the true means wouid be more iike that of the students in year 2009- 
2010. 

• Other mathematics measures that wouid contradict those in the tabie. 

• Speciai features that might have been in piace in 2009-2010 (e.g., changes in facui- 
ty, the curricuium, the admission standards, poiicy changes, etc.) to account for the 
iower mean and iarger standard deviation. 

• The range of the grades to be sure 4.0 was the maximum. 

• The percentage of students who had a 3.0 or higher math GPA and the number who 
passed Praxis ii (by the state criterion) and achieved 75% of what the top 10% of 
Praxis ii math scorers achieved on the test. 

• The comparison of the math grades of the graduates with math majors, not just the 
average of aii the graduates at the institution. 



Table 47b (repeated) 

Mean and sd GPA and Praxis II scores and pass rates in secondary mathematics 
for years 2006-2010 at Exemplar University 





GPA in upper level mathematics 
courses: math department mean: 
3.1 ; math department standard: 
2.75 


Scores on Praxis II: mathematics 
national mean: 540 
State cut score: 520 


Correla- 
tion: 
GPA- 
Praxis II 


Year of 
graduation 


Number gradu- 
ates/ 

number admitted 


Mean 

(0-4) 


SD 


Graduates 

N 


Mean and 
(pass rates) 
400-800 


SD 


Pearson 

R 


06-07 


15/20 


3.5 


.5 


15 


610 (84%) 


90 


.89 


07-08 


12/21 


3.4 


.4 


12 


590 (77%) 


95 


.70 


08-09 


14/22 


3.6 


.5 


14 


615 (85%) 


92 


.69 


09-10 


20/22 


3.0 


1.1 


20 


510 (47%) 


130 


.71 


10-11 


15/20 


3.5 


.5 


15 


610 (84%) 


95 


80 


Total 


76/105 


3.4 


.6 


75 


587 (75%) 


100 


.75 



Note: The faculty assessed the reliability of the GPA by drawing a sample of 30 students from the five-year 
period, and correlating the grades received on the odd lines on their transcript with the grades received on the 
even lines. The correlation was .78, providing support for a satisfactory level of reliability. ETS reports that the 
Praxis II examination in mathematics has a reliability of .83 for its norming group. The graduates of our program 
match well the characteristics of the ETS norm group. 

Are there problems serious enough for you to reject the conclusion that the program satis- 
fies Quaiity Principie / with regard to subject matter knowledge? 
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Probably not, because the evidence in favor of the conclusion is persuasive. There are two 
measures with reliability and validity determinations within the TEAC guidelines. With regard 
to the sufficiency of the two measures, each represents more than 75% of the maximum 
score available to the graduates. All the grades in math for 80% of the years were at or 
above 3.0 out of 4.0 and averaged at least 3.0 every year. The preponderance of the evi- 
dence (that is, 80%> of the time and for over 75% of the 76 graduates) is consistent with the 
claim that they know their subject matter. The graduates in most years exceeded on aver- 
age the grades earned in the math department, many of who were presumably majoring in 
math. The table offers no evidence that would indicate that the students are not competent 
in their mathematics knowledge. 
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Exercise 48 : Apple University’s claim for caring teaching skills: 

The faculty members at Apple University have claimed that their graduates leave the pro- 
gram with the teaching skills needed to perform well in their first school placement. The evi- 
dence they share in the Inquiry Brief to support their claim consists of three measures: 

(1) scores on the state tests showing pupil gains in performance on the state content stand- 
ards; (2) ratings received from the graduates’ first year principals; and (3) comparison of hir- 
ing rates of the program with other similar programs in the state. 

How credible is Apple University’s claim? What are some rival explanations for the findings 
reported below? Would you conclude there was sufficient evidence to support the claim the 
Apple faculty made? 

Apple University’s evidence 

Pupil gains on state tests . While the teacher education program at Apple University gradu- 
ates 200 new teachers every year, only 120 of them elect to accept positions “in-state.” 
Others take teaching positions in other states (n = 40) while others pursue life options in 
graduate school, in business, or in raising families (n=40). The state examinations of pupil 
achievement are administered only in grade and 5**^ grade. Of the 120 graduates who 
are teaching in state, only 40 are teaching at the 3'^^ grade or 5**^ grade level. The results of 
the pass rates for the pupils of those teachers are reported below in Table 46a: 

Table 48a 

Pass rates of students in 3*^^ and 5"^ grade of Apple University first-year graduates 



compared to si 


tate rates overall on the state curriculum test 


Grade level 


N 


Percent passing 


Percent failing 


3'"^ grade: Apple teachers 


25 


70% 


30% 


3'^ grade: Overall 


2,050 


55% 


45% 


5**^ grade: Apple teachers 


15 


65% 


35% 


5*'' grade: Overall 


2,200 


58% 


42% 



Principal survey . The Apple University faculty surveyed the principals of the 120 recent 
graduates of the program who were teaching. The faculty was fortunate in that of the 87 
schools in which the 120 Apple graduates were employed, 60 had principals who had either 
graduated from Apple University as undergraduates or had studied at Apple University for 
their administration credentials. This situation increased the likelihood that the principals 
would respond to the survey instrument. The results of the survey are disaggregated in Ta- 
ble 48b, below: 
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Table 48b 

Number of Apple-trained teachers rated excellent, satisfactory, and unsatisfactory 
on a survey of principals disaggregated by principals’ affiliation 
with Apple University 



Principals (N) 


Teachers rated 
as “excellent” 


Teachers rated 
as 

“satisfactory” 


Teachers rated 
as 

“unsatisfactory” 


Principals with previous 
affiliation with Apple 
University (n = 45) 


12 


30 


3 


Principals with no previ- 
ous affiliation with Apple 
University (n = 15) 


1 


9 


5 


Total number 


13 


39 


8 



Hiring Rates . Principals and directors of personnel in schools are very conscientious about 
hiring new teachers who have outstanding teaching skills. It is not enough these days to be 
smart or to have high grade point averages. Often candidates for teaching posts must pre- 
sent portfolios documenting their teaching practices, and on occasion they must demon- 
strate their skills by offering a lesson to pupils in the school. Hiring rates are quite signifi- 
cant, then, in attesting to the teaching skills of a program’s graduates. Apple University fac- 
ulty interviewed the directors of teacher placement at 20 sister campuses within the state 
with programs similar to those of Apple to determine what percentages of placement repre- 
sented the hiring rates of Apple’s graduates. Only 13 campuses were able to provide this 
information. The results are arrayed below in Table 46c: 



Table 48c 

Number of Apple University graduates and non-Apple graduates 
and percentage hired in and out of state 



Institution 


Number of 
graduates 


Percentage 
hired in state 


Percentage 
hired out of 
state 


Not hired 




200 


60% 


20% 


20% 


Sister 

institutions 


650 


55% 


15% 


30% 
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Exercise 48 , continued; Formulate your own response to the questions posed at the 
outset of the exercise: 

• How credible is Apple University’s claim? 

• What are some rival explanations for the findings reported in the tables? 

• Would you conclude there was sufficient evidence to support the claim the Apple 
faculty made? 

Use the space below for notes. 
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Auditing Data Tables 

Exercise 49 - Three sample tables: What can we learn from the data we 
report? 

Read the following three tables and discuss how you would interpret them. What more 
would you need to know in order to use this data for program improvement? (Following 
these tables are sample questions the audit team raised about the tables.) 



Table 49.1 : Percentage of students and faculty who agree that: 



Question 


Students 

Agree 


Affiliate Faculty 
Agree 


Ranked Faculty 
Agree 


1 . Program improvement 


89% 


97% 


1 00% 


2. Students are well-prepared edu- 
cators 


91% 


98% 


1 00% 


3. Students meet state standards 


70% 


98% 


1 00% 


4. Students use technology 


87% 


77% 


10% 


5. Students learn how to learn 


87% 


95% 


90% 


6. Students acquire multicultural 
perspectives 


87% 


81% 


50% 



Table 49.2 Average Scores for Work Samples 

The program prepares teachers for early elementary, elementary, K-12 (art, music), secondary, 
and special education. It has collected Work Sample data since 2006. What information would 
be useful to the program from data collected from the Work Samples? 



Criteria: Written Report 


Graduate 

average 


Undergrad 

average 


1 . Nine questions addressed in the report 


3.8 


3.5 


2. Professional appearance (professionalism) 


3.8 


3.4 


3. Unit and lesson planning (learning theory) 


3.9 


3.6 


4. Reflections (critical thinking) 


3.8 


3.9 


5. Use of formal & informal assessment data to drive instruc- 
tion (assessment) 


3.6 


3.5 


6. Evidence of P-12 student achievement (learning theory) 


3.7 


3.8 


7. Standard 1 : literacy (discipline knowledge) 


3.7 


3.7 


8. Standard 2: mathematics (discipline knowledge) 


3.8 


3.5 


9. Standard 3: standards and assessment (assessment) 


3.5 


3.8 


10. Standard 4: content area (discipline knowledge) 


3.7 


3.3 


1 1 . Standard 5: classroom & instructional management 


3.5 


3.6 


12. Standard 6: individualization of instruction (instruction) 


3.5 


3.3 


13. Standard 7: technology 


3.5 


3.3 


14. Standard 8: democracy (values) 


3.7 


3.3 
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Table 49.3: Student Teaching Assessment 



Proficiency 


Item / Question 


Midterm 

Rubric 

Score 


Final 

Rubric 

Score 


Communication 


Interacts respectfully by showing appreciation of 
the perspectives of others. 


3.3 


3.5 


Critical Thinking 


Analyzes student needs and utilizes conclusions 
to collaborate with resource persons to best 
serve students. 


3.3 


3.8 


Discipline 

Knowledge 


Presents accurate information in the content ar- 
eas he/she is teaching. 


3.5 


3.6 


Instruction 


Creates a positive and effective classroom 
learning environment for all students, using mul- 
tiple strategies. 


3.7 


3.8 


Assessment 


Analyzes the results of on-going content as- 
sessments to plan and adjust subsequent in- 
struction. 


3.1 


3.6 


Professionalism 


Presents as a professional in demeanor, ap- 
pearance, oral and written communications. 


3.8 


3.8 


Learning Theory 


Evaluates the unique characteristics of each 
student and chooses appropriate instructional 
strategies to address various learning styles and 
instructional needs. 


3.3 


3.6 


Values 


Communicates with students to encourage posi- 
tive behavior, such as cooperation with others , 
respect for rights of others, and character in 
challenging situations where ethical decisions 
are necessary. 


3.8 


3.8 


Technology 


Integrates computer technology to enhance les- 
sons, in record keeping, and in communication. 


3.1 


3.6 
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^Answer Key to Exercise 49, Auditing Data Tabies 

Table 49. 1 - Possible questions asked by auditors considering the reported data. 

1. Where do these data come from? 

2. How many students, affiliate faculty, and ranked faculty are represented by these 
percentages? 

3. When did these people respond to these items? What were responses like over 
time? 

4. On what basis did participants “agree” that: ( 1 ) there was program improvement, 
(2) students are well prepared, (3) students meet state standards, (4) students 
use technology, (5) students learn how to learn, and (6) students acquire multicul- 
tural perspectives ? 

5. What does “program improvement” (row 1) mean? 

6. What percentage of agreement does faculty want for a question ? What response 
did the program faculty make to the 70% of students in agreement that they meet 
state standards? What response was made to the 77% of affiliate faculty and 10%> 
of ranked faculty in agreement that students use technology? 

7. What does “use technology” (row 4) mean? 



Table 49.2 - Possible questions asked by auditors considering the reported data. 

1. What is the scale for the responses? 

2. How many graduate students? How many undergraduate students? 

3. How do students from the various program options respond to this assessment? 

4. What year(s) are being reported? Are there differences over time for each item? 

5. What are the high/low scores for each item? What is the standard deviation of the 
scores? 

In response to a Clarification Question, the program responded that the Graduate Aver- 
age was compiled from scores of 12 students: 1 in 2006, 3 in 2007, 6 in 2008, and 2 in 
2009. The Undergraduate Average was compiled from scores of 6 students: 3 in 2007, 
2 in 2008, and 1 in 2009. The program includes preparation for early elementary, ele- 
mentary, K-12 (art, music), secondary, and special education. Work Sample data have 
been collected at least since 2006. What information would be useful to the program 
from data collected from the Work Samples? 

Table 49.3 - Possible questions asked by auditors considering the reported data. 

1. What is the scale for the responses? What are the high/low scores for each item? 
What is the standard deviation of the scores? 

2. How many students are included in the averages? Are the students graduate or un- 
dergraduate? What program option? Are there differences in the ratings for students 
in different program options? For graduate and undergraduate students? 

3. Who is scoring the student teaching assessment? Is this a compilation of ratings by 
cooperating teachers and university supervisors? How do scores from the cooperat- 
ing teacher and university supervisor differ for each item? 

4. What year(s) are being reported? Are there differences over time for each item? 
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